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REPLIES OF PSYCHOLOGISTS TO A SHORT QUES- 
TIONNAIRE ON MENTAL TEST DEVELOP- 
MENTS, PERSONALITY INVENTORIES, 

AND THE RORSCHACH TEST 


ARTHUR KORNHAUSER 
Bureau of Applied Social Research, Columbia University 


In Арки, 1944 a brief questionnaire was sent to 85 selected 
specialists on mental tests whose views might be considered 


representative of highly competent thought in that field. 
Seventy-nine completed blanks were returned (93 per cent). 

The first part of the question-blank asked a few questions 
about the practical value of intelligence tests. This material 
will be summarized in a later report in this journal as well as 


in more popular form elsewhere." 
The second half of the questionnaire, to be reported here, 


Contained somewhat more technical questions. These were 
Intended for report within the profession only, not for the 


general public. 
Both the formulation of the questions and the selection of 


the expert panel were based upon personal conferences with 
Six specialists who served as advisors. The final list of experts 


represents the pooled judgment of these advisors.” 
— 
! The set of questions on intelligence testing constituted one of a series of “polls 

ОЁ experts” which aim to ascertain and report to the public the conclusions of a 
Cross-section of leading authorities on questions in their special fields. A continuing 
Project of this kind, it is believed, may help reduce the lag of public thinking 
Pehind the views of the well-informed. The polls are intended for prompt pub- 
ication in а mass-circulation magazine. While publication arrangements have been 
elayed during the initial stages, plans are now completed for having the poll 
Teports appear monthly in The American Magazine. Sh 1 
е cooperation of the mental test authorities who participated is gratefully 
acknowledged. In addition to those in the following list, three others requested 


that their names be not listed. 
Dr. Dorothy C. Adkins Dr E К. Киш 
Dr fane Anastasi Dr. Irvin, Lo Ea 
ose G. Anderson Comat, "& M. Louitie 
" " . " 


Dr. Grace Arthur 
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A summary of responses to the second set of questions 


follows. 


Question 1 


In the further development of mental ability testing for prac- 


tical use in schools and in business, 


do you think most will be 


accomplished if psychologists concentrate on measuring sepa- 
rate intellectual factors or if they continue to emphasize the 


measurement of “general” intelligence? 


Separate factors .................. 


General intelligence 


Both “separate” and “general” checked 
Other answer or no answer 


In addition to the seven who checked both, there are 9 
others whose comments suggest that both types of emphasis 
are desirable. Even with these included, an overwhelming 


"A ee ee ee EE oe 


Major Roger M. Bellows 
Dr. George K. Bennett 
Lt. Col. Robert G. Bernreuter 
Dr. Marion A. Bills 

Dr. Walter V. Bingham 
Dr. Robert A. Brotemarkle 
Dr. Andrew W. Brown 
Dr. Harold E. Burtt 
Dr. Herbert S. Conrad 
Capt. Clyde H. Coombs 
Dr. Albert B. Crawford 
Dr. Edward E. Cureton 
Dr. John G. Darley 

Dr. Walter F. Dearborn 
Dr. Edgar A. Doll 

Lt. Comdr. Jack W. Dunlap 
Dr. Beatrice J. Dvorak 
Dr. Harold A. Edgerton 
Dr. Max D. Englehart 
Dr. Alvin C. Eurich 

Dr. Warren G. Findley 
Dr. Frank N. Freeman 
Dr. Douglas H. Fryer 
Dr. Henry E. Garrett 
Lt. Col. J. Guilford 

Dr. Harold O. Gulliksen 
Dr. V. A. C. Henmon 
Dr. Edwin R. Henry 

Dr. Gertrude Hildreth 
Dr. Karl J. Holzinger 
Dr. Carl Hovland 
Comdr. John G. Jenkins 
Dr. H. M. Johnson 

Dr. Truman Kelley 

Dr. G. Frederic Kuder 


Lt. Col. Arthur W. Melton 
Dr. Arthur S. Otis 

Dr. Jay L. Otis 

Dr. Donald G. Paterson 


Lt. Col. Robert T. Rock 
Dr. Carl R. Rogers 

Dr. Phillip Rulon 

Capt. Edward A. Rundquist 
Dr. David Segel 

Lt. Col. Morton A. Seidenfeld 
Lt. Col. Laurance F. Shaffer 
Dr. C. Shartle 

Dr. John M. Stalnaker 

Dr. Ruth Strang 

Dr. Percival M. Symonds 
Dr. E. L. Thorndike 

Capt. R. L. Thorndike 

Dr. L. L. Thurstone 

Dr. Joseph Tiffin 

Dr. Lewis E. Terman 

Dr. Herbert A. Toops 

Dr. M. R. Trabue 

Dr. Arthur E. Traxler 

Dr. Ralph W. Tyler 

Dr. Morris S. Viteles 

Dr. David Wechsler 

Dr. Frederic Lyman Wells 
Dr. Edmund G. Williamson 
Dr. Ben D. Wood 

Dr. Herbert Woodrow 

Dr. Wayne Wrightstone 


MENTAL TEST DEVELOPMENTS 5 


majority of the 79 experts answer in favor of “separate 


factors.” 
Some typical comments from those who say both ' 


factors” and “general intelligence”: 


‘separate 


Both are important. Developmental work in “separate fac- 
tors” should probably be given higher priority. 

“Separate factors” for a period until we find what it’s all 
about. I don’t believe tests of these “separate factors” will 
ever supplant entirely the test of general ability. 

I think that there is still a definite use for the “general intelli- 
gence” test, but the measurement of “separate factors” should 


also be made. 

Several others simply say, “Both needed.” One interest- 
ing belief in both is expressed in these words: “ ‘General in- 
telligence’ tests for children and group factor tests for adults.” 
Another suggests that it is a question of the time available for 
testing: If only one hour, a general intelligence test; if three 
or four hours, tests for separate factors. — + 

Five of the psychologists explicitly reject “general intelli- 
gence” measurement altogether. They say, for example: 


“General intelligence” is а hodge-podge of several relatively 


independent group factors. | : 
The concept of “general intelligence” should be entirely dis- 
carded. р 
“General intelligence” is like what Henry Ford said about 
history. 


In advocating major attention to separate functions, two | 
replies particularly stress testing for the three abilities—verbal, 
numerical and spatial or mechanical; seven point to the special 
usefulness of separate ability tests for industrial or other spe- 
cific purposes; three dissociate their belief in measuring sepa- 
rate abilities from any particular “factor analysis” methods 
or any particular classification of abilities. 

On behalf of continued emphasis on the measurement of 
“general intelligence” the following comments are made: 


eral intelligence” tests has been demon- 
d in their widespread use particularly in 
]ue of measures of “separate fac- 


The value of “gen 
strated as is indicate 
schools; the validity and va 
tors” have yet to be shown. 
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With criteria as undependable as they now are, tests of gen- 
eral ability do as much as can be expected and tests of separate 
factors represent excessive refinements. 


In respect to measuring isolated functions, attention 15 


called, in four or five answers, to the limitations of this pro- 
cedure. For example: 


All so-called mental abilities seem to be intercorrelated. 
"Therefore you cannot test an isolated factor if you try. 


The idea of test purity is far from clear. At best factorial 
composition always involves a “potpourri” factor. 


"Separate factors" in the strict sense cannot be now measured, 

but approximations can be made. 

A point of some interest with respect to the tabulated 
replies to this question is their relationship to the age of the 
respondents. ; 

The median age of those who answered “general intelli- 
gence” either alone or together with “separate factors” is 56 
The median age of all others is 42. Looking at the matter 
in the other direction, of the 28 persons 50 years of age and 
over, 29 per cent answered "general intelligence"; of the 44 
persons under 50, only 9 per cent answered in this way. It 


almost looks as though "general intelligence” is becoming a” 
old man's concept! 


Question 2 


In the field of personality testing, how satisfactory or helpful 
for present practical use do you consider: 


a) Personality inventories and questionnaires (such as those 


of Bernreuter, Bell, Humm-Wadsworth, etc.)? 
(b) The Rorschach test? 


Question (a) Question (0) 
N % No. % 
15 0 0.0 
13.5 12 20.0 
36.0 17 29.0 
33.0 13 220 
16.0 17 290 
Qualified; unclassifable* ............ 6 1099 5 ж 
No answer or don't know ............ 4 15 


* Of the 8 qualified responses on i ] un- 
f: ble, 2 1 question (2а), 5 tend to be favorable, 
and 2 pede i Of the 5 qualified replies to question (2b), 3 are favorable 
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| The ratings of the Rorschach test tend t ive a flatter dis- 
tribution than the other. A slightly higher percentage of the 
psychologists consider the Rorschach “moderately satisfactory” 
but a markedly higher percentage also rate it as “highly un- 
Satisfactory.” There is almost no correlation between the 
ratings assigned by the respondents to personality inventories 
and to the Rorschach tests (7 = .11). 
When the respondents are classified into clinical and non- 
clinical (according to their statements about their own principal 
types of work), some interesting differences appear in the re- 


sponses to the above questions. 


Personality inventories Clinical Non-clinical 
Rated: Highly or moderately satisfactory ...- 255 пок 
ў 34, 


Doubtfully satisfactory ---+-++*- Е 
Highly or rather unsatisfactory 39.5 55.0 


ПОРЕ 100.0% 100.0% 
Total per cent .. (N28) b 
Rorschach Test Clinical Non-clinical 

Rated: Highly or moderately satisfactory -----++-- 38.0% 110% 
Doubtfully satisfactory .-++++++ А 29 100 
Highly or rather unsatisfactory .. 33, y 

AN 100.0% 100.0% 

Total per cent ...... 27 (525) 


ts are somewhat more 
Their opinions differ 
larly in respect to the 


It is clear that the clinical psychologis 
favorable toward both types of tests. 
from those of the non-clinicians particu 


Rorschach. | 
Similar tabulations have been made comparing the psy- 


chologists who state that a principal part of their work has 
dealt with personality tests and those whose work has not been 


Principally in this field. The results are as follows: 
Work with Personality Tests 


‚‚ Personality Inventories a) oo 
ighly or moderate satisfactory See | Behe 40. 
oubtfully satisfactory 555577 37 51 
ighly or rather unsatisfactory -+-+ i (N =30) (N=35) 
Work with Personality Tests 
Е Rorschach Test ithe 
Highly or moderately satisfactory $^ 


Oubtfully satisfactory ..«««:000 0t 
ighly or rather unsatisfactory «««-:00000007077 4 Rees 
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Comments Regarding Personality Inventories 


The most frequent comments on question 2a are those 
which point to the clinical and qualitative value of the blanks 
as contrasted with their quantitative use for purposes such as 


selection of personnel. Under this heading come such remarks 
as the following: 


Useful in locatin 


g foci for further exploration and counseling, 
1.е., qualitativel 


y rather than quantitatively. 
No use for selection, possibly some clinical value. 


I believe that the personality test used as a clinical tool or 
used under conditions where there is full cooperation of ue 
testee and the tester can be very illuminating and practical S 


useful. We have not found that they give us the correct in- 
formation when used as a selection tool under ordinary cir- 
cumstances, 


Moderately satisfactory for clinici 
for industry. For industry, subjec 


Most useful in counselling as points of departure, securing in- 
terest on adjustmen indi 


be referred to a ps 


ans; highly unsatisfactory 
t will not *come clean. 


A second group of comm 


: or 
ene ents has to do with the need f 
validation an 


d further research: 


, primarily to inadequate standard- 
lidity. 


if 

record to this effect, Р а 
Th : TE 
ir pra frm a PB be have been helpful. Ту 
He Ing stan ы 
in et » Using ard norms, they are not helpf 
на and опе- 
helpfulness in » i 

h : ade" tests which are pre- 
sumed to measure “traits” which clinicians presumed to exist- 
Navy have been able to 


half years of work i 


\ 


кь —A 


— ——— Du — PG PITT 
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Other comments call attention to the value of the person- 
ality blanks insofar as they are competently and cautiously 


interpreted. Examples: 


If used by qualified person. Should never be depended on for 
individual diagnosis without careful check. Of some value for 
research. 

A very great deal depends on the skill and judgment of the 
person making use of the results. 

In the hands of competent clinicians such devices appear quite 
useful—used with other data. For most counselors, who may 
not add salt, better counselling in regard to personality prob- 
lems will be produced without such inventories and ques- 
tionnaires. 

tention to the differences in value 


Several replies also call at 
As one reply puts it: 


among the various inventories in use. 


There are some 500 personality tests, most of which are of 
little or no value as measurement devices. A few, probably 
not more than a dozen, could be recommended for experi- 


mental use in a testing program. 


In the few comments regarding particular blanks, the Bell 
and the Minnesota multiphasic inventories receive favorable 
mention, the Humm-Wadsworth unfavorable, while the Bern- 


reuter receives both praise and disapproval. 
Scattered comments on other points of interest about per- 


sonality questionnaires are these: 


They are difficult to improve beyond the present level. It 
will take a first-rate genius to make any great improvement. 
A lot of effort has been expended lately, with little progress 
resulting. MM 
Moderately satisfactory at extremes of distribution. Prin- 
cipal value impresses me in terms of serious deviation from 
norm rather than as absolute score values. 

The psychologically sophisticated person who has some motive 
for haking a good impression can consciously distort results; 
disturbed person doesn’t know the true answers with respect 
to himself. х 


Such tests should always be supple 


i 5 
as observations. anecdotal records, € E [ues 
Personality inventories are often excellent aids in "screening" 
detailed observation and study. 


individuals for further 


mented by other data, such 
and projective techniques. 
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Comments Regarding the Rorschach Test 


The quantitative replies to Question 2b were tabulated 
above. The ratings are not greatly different from those per- 
taining to personality questionnaires save that there is an in- 
crease in the number of “Highly Unsatisfactory” ratings in 
contrast with the “Rather Unsatisfactory,” and there is a 
decided increase in the number of “Don’t Know” responses. 

The comments with respect to the Rorschach test are 
notably more vigorous. There are numerous references to 
“cultism” and “overselling,” and even more frequent specific 
criticisms concerning the lack of validation. On the other 
hand, a considerable number of the psychologists believe that 
the Rorschach has value if used clinically by adequately 
trained persons. Since the Rorschach technique is so much in 
dispute, it is worth while reproducing a considerable number 
of evaluative comments. They are grouped below into the 
two broad divisions just mentioned, plus a miscellaneous set 
of comments. The parentheses after each quotation contain 
the rating assigned the test and also indicate whether the 
respondent considers himself a clinical psychologist or not. 


A. Rorschach Test of More or Less Value Used Clinically by 
Trained Persons 


Dangerous for amateurs. A valuable instrument in the hands 


of a psychiatrist adequately trained in its use. (No rating; 
clinical.) 


I feel this is of value as a strictly clinical instrument in the 
same way that free association is, but any attempt to objectify 
scoring of it appears to lead to invalid results. (Doubtfully 
satisfactory; clinical.) 

The Rorschach has already demonstrated its value in clinical 
usage. The increasing research by “courageous heretics” on 
modified Rorschach techniques may be expected to produce 
instruments of considerably more merit than аге yes-and-no 
inventories. (Moderately satisfactory; clinical.) 

In the hands of a few well-trained experts the Rorschach test 
may be “moderately satisfactory,” but it requires too much 
highly specialized training under the right supervision to de- 
velop the reliability needed in practical work. (Doubtfully 
satisfactory; clinical.) 

If one sticks to the few basic categories, the Rorschach is valu- 
able. I tend to doubt the validity of the detailed analyses 
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some Rorschach experts are able to make. (Moderately sat- 
isfactory; not clinical.) | 


В. Lack of Adequate Validation; “Cult,” etc. 


I think more systematic research and less cultism could pro- 
duce something of value in this particular projective tech- 
nique, as would be true of any projective technique. (No 
rating; clinical.) 
Do not feel “expert” on this item. This test smacks too much 
of mystery and “cultism.” Also rather too esoteric in rami- 
fications for sound scientific appraisal. (Moderately satisfac- 
tory; clinical.) 

There is need for an empiric 
am impressed by the extent to which its 
a priori in terms of some semantic scheme. 
isfactory; clinical.) 
Highly promising b 
conclusions can be warrante 
entifically unsound; but some 800 


(No rating; clinical.) . 

Too subjective; clinical signs employed are shifted from study 
to study. Still in the experimental stage and should not be 
used for practical purposes as yet. (Rather unsatisfactory; 


clinical.) | 
Аз a diagnostic instrument its value is entirely unproved and 
the Rorschach workers are going about its validation the 
wrong way: Too much cultism and intuition and too few 
cold facts! (Highly unsatisfactory; not clinical). | 
There has been grossly inadequate validation of the claims for 
the Rorschach. (Doubtfully satisfactory; not clinical.) | 

So time-consuming and subjective as unlikely to contribute 
much that a skillful interviewer, would not obtain more 
promptly by direct means. Highly unsatisfactory; not 
nical.) кө ТЗ 
Found utterly useless for predicting success in training of 
aviation pilots. (Highly unsatisfactory; not clinical.) 
Those who use the Rorschach seem always to fall under the 
spell of the special language t ey have developed and to be 
more interested in assigning names than in making any ex- 
tensive and critical investigation of the validity and reli- 
ability of their basic concepts- ( Doubtfully satisfactory; not 


clinical.) 


al validation of this technique. I 
validity is assumed 
(Doubtfully sat- 


ut needs much further research before 
d. Much of present work is sci- 
d leads have appeared. 


C. Other Comments 
When, as in some hands, the Rorschach test proves useful I 
attribute it more to the good sense of the ie than to the in- 
strument. (Rather unsatisfactory; clinical.) 
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This test still leaves much to be desired but is certainly 6 
step in the right direction. If it could be more easily scorer 
and more objective it would be the most effective instrumen 
I know of for clinical measurement. (Moderately satis- 
factory; clinical.) ч 

From the standpoint of the “buyer” it is worth about 15% of 
the time one can spend on an individual examination. For 
the work I do, would practically always want it ог a sub- 
stantial equivalent; for dynamic purposes its potentialities are 
less than those of T.A.T. (Moderately satisfactory; clinical.) 


The Rorschach tests is highly promising. Group sinis 
tration techniques should make it more widely applicable. 
(Moderately satisfactory; not clinical.) 


А good idea poorly carried out. (Highly unsatisfactory; not 
clinical.) 


Question 3 
What do 


you consider the most promising mental test de- 
vel 


opments for research students to devote themselves to 
uring the years after the war? 


; : : er 
Most of the replies fall into a few broad categories, m 

which responses are classified below. (In considering m 

numbers of replies in different classes it should be noted th? 


à > ег 
many respondents listed several ideas; hence the total numb 
of suggestions far exceeds the nu 


Most frequently mentioned 
new tests, especiall 


mber of persons answering.) 

are needed developments 9 
y tests of emotion and personality traits. 
Thirty-six of the 79 psychologists point to work on new tests; 
27 of these indicate tests in the field of personality. This 
result may have been influenced in some degree by the fact 


that the immediately preceding questions pertained to person- 
ality blanks and Rorschach tests. 


Illustrative Suggestions Regarding Development of New Tests 
Independent measures of ability; 


Tests of mental development that evaluate objectively the 
higher mental processes (as in the Eight-Year Study of 
the Thirty Schools Experiment of the P.E.A.). 

Better non-language performance tests for children and adults. 


Better materials, individual and group, for exploring aptitudes 
of gifted children. 


Mental tests for adults, especially older persons. 
Short tests for industrial use. 


specific aptitude tests. 
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Culture-free tests of general ability. 

Tests for special groups—bilingual, blind, deaf, etc. 

Wisdom, thinking, judgment, etc., as distinct from mental 
alertness. 


Development of tests for such traits as perseverance, ability 
to supervise (leadership), emotional and social maturity. 


bjective personality tests; indirect scoring so that subject 


doesn't know its real purpose. , 
Projective techniques as personality tests; 


projective tests. 

Personality tests using operationally defined concepts tied to 
particular fields and giving up the magic of such alleged 
traits as “extroversion,” “dominance,” and the like. 

Measurement of personality factors and of the as-yet-un- 
measured but important intellectual factors. , , 

One important area in business is to measure “drive” in pros- 


pective executives. —— . Я T 
Tests for specific personality traits for which good criteria are 


available. 


Measurement of “basic human drives." 

Useful and readily scorable interest inventories. —. 

Interest measures through a wider range of occupational, edu- 
cational, and avocational activities. | | 

Achievement tests through a wider range of life and job 


situations. Я М 
ent that yield sub-scores indi- 


Tests of educational developm 
f intellectual development. | 


cating specific aspects О 
Trade and proficiency tests geared to specific ey ork ' 
ests to measure achievement that will actually be functiona 
in normal life, e.g- homemaking, consumer science health, 
marriage, child development, social attitudes, labor rela- 
" Ж " 
tions, propaganda analysis, etc. 


Mentioned next most often (by 18 respondents) is the need 
lor work on criteria and the carrying On of validation studies 
to ascertain the relation of particular tests to criteria. 

Illustrative Suggestions Regarding Validation Studies 
and Criteria 


n on large representative 
ja, especially with adults 


objectification of 


n validatio 


More concentration O 1 
criteri 


Samplings against adequate 
against vocational success. 

The predictive values of speci 
in practical tasks. From these s 
develop data regarding the “types 
cess in “families” of occupations © 


fic tests for specific performances 
pecifics it will be possible to 
» of tests that predict suc- 


г activities. 
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Factor analysis of abilities and of criteria and establishment 
of satisfactory inter-relationships among the tests and criterion 
factors. 


Decrease emphasis upon new test construction; increase 
emphasis upon the development of adequate criteria and 
validation and cross-validation of tests of special aptitude and 
other predictors against such criteria with samplings of ac- 
ceptable size. 


Development and evaluation of realistic criteria for: (a) occu- 
pational successes in given job titles, (b) occupational success 
as general occupational adjustment, and (c) general social 
and personal adjustment. Of the two needs (work on tests 
and on criteria) it is considered that the criterion side should 
be given most emphasis. 

By all means investigators should work as hard on good 
methods of evaluating proficiency on the job as they do on 
tests. As an end in itself, this has most salutary effects, but 
it is essential if one is to validate selective tests adequately. 


The one remaining category into which many suggestions 
fall (17 responses) pertains to studies aimed at analyzing and 
interrelating the component factors of ability—either by fac- 
torial methods or otherwise. 


Illustrative Suggestions Regarding Factorial Studies 


Application of factor analysis results to the construction oi 
differential aptitude batteries, followed by standardization О 
such batteries on a wide range of schools and occupations. 

Identification and measurement of separate factors. Con- 


struction of tests which will measure such factors as inde- 
pendently as possible. 


Psychological analysis of separate or group factors to supple- 
ment or replace the mathematical analysis now so much em- 


phasized. In recent years we have had an orgy of statistical 
analyses. 


Isolation of meaningful complexes or factors. 
Refined experimental work on the isolation of mental abilities- 


Fundamental research on the best reference variables of intel- 


lectual and temperamental aspects of personality is badly 
needed. 


In addition to the above types of reply, the question elicited 
a variety of other individual answers. The content of a пит” 
ber of these may be suggested by a mere listing of topics: — 
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Research on the interview. 
Attention to traits which are not measurable. 


Analysis of job requirements. 

Attitude inventories. 

Relationships between successive annual increases 
of intelligence. 

Changes of intelligence with age. 

Construction of prediction tables. 

Reliability studies. 

More adequate ad 
representativeness of samples. 


A few further points of some interest wh 
covered in the above categories are presente 
quotations: 


ult norms; greater attention to 


ich have not been 
d in the following 


at will measure all of 


a general battery that v 
factors and which can be secured 


The development of 
ing the entire occupational 


the occupationally significant 


for groups of occupations coverin i 
range. Such a testing technique 1s needed for occupational 


counselling. The findings with respect to groups of occupa- 
tions requiring similar abilities would also have an important 
earing on curricular development. Accomplishment of this 
task would demand cooperation among various research 
groups. 
Developments designed to encompass all major aptitudes as 
Opposed to particularity of appraisal, ie. composite descrip- 
tion of the complete person. e need test batteries stand- 
ardized on the same sample with interrelations and differential 
validities. 
The most urgent need is to try many tests of various kinds 
Оп the same people. . . - The establishment of unique human 
Profiles is our most urgent need. а . 
Relation of responses as elicited by inventories and question- 
naires to variations in behavior under different environment 
and changes accompanying education and training. 
The use of cumulative records of comparable test data 
throughout the school and early employment life of the indi- 
vidual, with equal or greater interest on cumulative anecdotal 


records of actual behavior. 


CIVILIAN TESTING IN THE QUARTERMASTER 
CORPS 


W. C. KVARACEUS anv W. №. DUROST 
Civilian Testing Section, ASF 


AND 
R. F. McCLELLAN 
Office of Quartermaster General 


Tue QUARTERMASTER Corps is one of the several supply 
services in the United States Army. The duties of the 
Quartermaster Corps comprise the initiation, procurement, 
supply, and maintenance of all articles and equipment needed 
by every soldier or necessary to the administration of the 
United States Army with the exception of the weapons with 


Which the soldier fights and certain classes of transport. In 
clothe some nine million men quar- 


the Army maintains 22 Quarter- 
Depots in the United States 
rker to assist in this vital 
80,000 civilians are em- 


order to supply, feed and 
tered in both hemispheres, 
master and Army Service Forces 
and draws heavily on the civilian wo 
Phase of the war effort. Roughly, 

Ployed in some 300 different jobs in these depots throughout 
the country. These civilian jobs range from the highly skilled 
technical and professional positions to those of unskilled 


aborers, ' 

With the advent of the war Quartermaster Corps, like other 

technical services, expanded to fantastic proportions in the way 
» 


of a world-wid ly organization. This tremendous growth 
ix ganiza 
ide supp'y ds of workers and presented 


emanded the hiring of thousan 1 
ining, and employee relations. 


many problems of assignment, tral 
the Quartermaster Gen- 


Lieutenant General E. B. Gregory, wks 
ега], was quick to recognize the basic management principle that 
Since civilian employees 


г | 

€sults are achieved through people. T А 

Comprise а large percentage of the total personnel in the in- 
:on of the Quartermaster General, 


s Ў AP RS 
tallations under the jurisdictio 
17 
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the accomplishment of the mission of the Quartermaster Corps 

depended, to a very large extent, upon the effective manage- 

ment and utilization of civilian personnel. As one means to- 
ward this goal, the Quartermaster General, through Colonel 

Eugene G. Mathews, Chief of the Civilian Personnel Branch 

of the Office of the Quartermaster General, in close coopera- 

tion with the Civilian Personnel Research Sub-Section, Per- 
sonnel Research Section, Classification and Replacement 

Branch, Office of the Adjutant General, has encouraged the 

proper use of psychological tests. It was felt that these tests 

could provide concrete evidence concerning employee knowl- 
edge, aptitudes and skills for use particularly as an aid to 

Placement, Training and Employee Relations activities. AS # 

result of much thinking and considerable experimentation 1” 

a number of depots, a Civilian Testing Section has been ser 

up within the Civilian Personnel Branch, Personnel Division 

. at ООМС to: 

a. Encourage the use of ability, skills, and aptitude test- 
ing in all Quartermaster and Army Service Forces x 
pots employing civilians, and to advise in the 65122; 
lishment of a Testing Section as a component part 9 
the personnel organization. " 

b. Coordinate and standardize all testing activities pe 
rently being conducted and proposed in all Quart? 
master and Army Service Forces Depots. 

c. Render technical and staff assistance to all Qu D 
master and Army Service Forces Depots through A 
issuance of a testing manual that will serve as the © i. 
guide in the use of tests and testing materials and SP 
cific Quartermaster testing policy and procedure. 

f. Compile for the Quartermaster General such pro 
reports on testing as may be requested. 


Cooperation of Office of The Adjutant General 2 

At the request of Colonel Eugene G. Mathews, Chief cef 

the Civilian Personnel Branch, Office of The Quartermas ef 
General, two technicians! were assigned to the Quartermas 


+ tim 
+Dr. W. С. Kvaraceus and Dr. W. N. Durost, who were attached at this 
to the Field Staff, Civilian Personnel Research Sub-Section, AGO. 


iarten 


gresi 
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Corps to assist in planning and setting up testing programs 
in selected depots and at the headquarters level. This was to 
Provide a background of experience on the basis of which to 
Set up service-wide testing, if the experimental program showed 
any worth-while promise in solving some of the personnel prob- 
lems facing the Placement, Training and Employee Relations 
officials. The office of the Adjutant General already had con- 
structed tests of general intelligence, clerical and mechanical 
aptitude, and knowledge and skills and had been given clear- 
ance for use of all testing materials prepared by the United 
States Employment Service. Machinery had also been set up 
Within the Civilian Personnel Research Sub-Section of the 
Adjutant General's Office to construct additional tests when- 
ever the need for such tests was shown. In general, the de- 
velopment of a systematic and comprehensive service-wide test- 
ing program throughout the Quartermaster Corps has been 
done with the active assistance and cooperation of the Office 


of the Adjutant General. 
Trial Testing in Selected Depots 


Some testing already was going on in several depots? before 
technical assistance was procured from the Office of the Ad- 
Jutant General. This testing was usually an adjunct of either 
the placement activities, the training program, or the employee 
relations activities. Often, the testing was spotty and hap- 


hazard, and seldom was a qualified full-time or even part-time 
The testing activities in these depots, 


technician in charge. : 
f the fact that some assistance 


Owever, revealed an awareness О 
Could be obtained in the personnel program through the wise 


Use of psychological tests. 
Personnel technicians from AGO 


€pots to set up testing activities. i 
technicians with adequate professional and clerical staff were 


recruited to head the program in the local installation. In 
another depot, the testing activities already had been set up 
under the Placement Branch. In one of the new installations 
Se 


hia Quartermaster Depot which, under its own 
hensive testing program prior to headquarters 


visited two Quartermaster 
In each instance, qualified 


initig Credit is due the Philadelp 
lative, had activated a compre 
anning, 
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Testing was placed under Training. In the other, Testing 
was established as a separate unit coordinate with Placement, 
Training, and Employee Relations. The head of this separate 
testing unit was made immediately responsible to the Civilian 
Personnel Officer. The latter type of organization was finally 
recommended and adopted as most promising if a depot-wide 
program were to function with the maximum effectiveness. 

As soon as the staff was recruited in a depot, attention was 
turned to the application of testing as an aid in the solution 
of operating problems. In one of the initial depots it was 
found that important reassignments and promotions were 10 
be made within the Shoe Inspection Section. The local test- 
ing technician, aided by the AGO representative, prepared а 
battery of tests which was given to all shoe inspectors attached 
to the depot. This battery included a learning ability test 
a shoe inspection information test which was constructed for 
the purpose and published by AGO, a man-to-man rating scale; 
and an activity preference questionnaire, also specially СОП” 
structed. After the tests were given and the data summar!Ze 
for the operating officials, further assistance was rendered 1” 
utilizing the test results in specific personnel actions. For oe 
ample, the five best all-round men were selected from which 
one was later chosen for a special assignment. ; 

, In another depot, file clerks were tested with appropriate 
instruments to discover those clerks whose filing skill was low 
and who were largely responsible for “messy filing conditions: 
Those file clerks who showed limited alphabetizing skill, pur 
who did reveal high learnability, were assigned for training, 
the file clerks who lacked aptitude for this job were re-test? 
with other clerical batteries and were reassigned to jobs 
which they showed more promise. In still another depo? 
where considerable difficulty had been experienced in the Fisc? 
Branch due to numerous errors in arithmetic processes F3 
fiscal clerks were given arithmetic tests to discover the P” 
viduals who might be largely responsible for the recurring 
arithmetic errors. Again, according to the test findings, cler 
either were retrained or reassigned. At the same time, testi 
of all incoming employees was started immediately as ар й 
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in the assignment of personnel to specific jobs at grade as 
determined by the Civil Service Commission. 

The keynote of the field service was the development of a 
testing program which would serve as an aid to solving depot 
Problems and, at the same time, demonstrate the potential 
value of test results in guiding personnel action. These trial 
testing programs rapidly expanded and became an integral part 
of the depot organizations. With this experience and the con- 
Viction that the wise use of tests could materially aid these and 
other depots, the establishment of a Testing Section at the 
headquarters level, Office of the Quartermaster General, was 
accomplished. The purpose of the newly established Testing 
Section was to coordinate, encourage, and advise in the estab- 
lishment and functioning of testing sections throughout depots 
under the jurisdiction of the Quartermaster General. 


The Contribution of Testing to the Total Personnel Program 


On the basis of the experience in these depots, testing was 
conceived as a service (staff) function standing in relation to 
the Civilian Personnel Officer in much the same way that Depot 
Control stands to the Commanding Officer. Tests could be 
helpful in that they provide objective evidence in the form of 
test scores upon which personnel action could be based. Some 
of the personnel actions to which testing was found to make 
à notable contribution were as follows: 

Placement of incoming personnel. Although the Civil Ser- 
Vice Commission reserves the right to certify employees as to 
Brade, it does not attempt to specify to what specific duties 
Within a given grade an individual is to be assigned. This 
leaves the local depot considerable latitude in placing new per- 
Sons on jobs for which they are best suited. The local depot 
Itself must test new people if test scores are to be made avail- 
able for most effective placement purposes. A variety of tests 
may be used for this purpose, but basically most Quartermaster 
Depots found that a limited battery of aptitude and achieve- 
Ment tests could serve most purposes satisfactorily. 


Reassignment of personnel at grade. Reclassification of 
i regardless of the direction 


Personnel is a very serious busin 


me: > 
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of the change, whether it be promotion or demotion. ‘Testing 
can do much to support such personnel action by demonstrat- 
ing the presence or the absence of the desired skills, knowledge, 
or abilities. 

Selection of personnel for training. Far too much of the 
training being carried on in the operating installations was 
found to be haphazard in the sense that a general order was 
issued to train persons in some specific area such as the use of 
the War Department Shipping Document or military corre- 
spondence, without knowledge of which of the persons selected 
for training already knew the material to be covered by the 
course. In the case of those whose knowledge was incomplete 
there was no evidence as to the specific areas where gaps 1” 
knowledge existed, so that the subsequent training was neces- 
sarily on a general rather than a selective basis. By the 
judicious use of specific information tests in such situations» 
three things were accomplished. (1) Those with a mastery 
of the information sufficient for the needs of their job wet 
excused from training. (2) The training of those lacking SUC 
basic knowledge was justified to their supervisors on the basi 
of objective evidence. (3) The training was directed to meet 
the areas of the greatest need. Following training, retests T° 
vealed the extent to which training had been successful 1” 
imparting basic information. Note should be made here that 
the failure to get information across to a group may be due t? 
a variety of causes, Testing alone will not reveal the reaso” 
for failure, but only its existence. Р 

Substantiation of claims of supplementary or higher skill. 
Many manpower utilization programs listed employees’ supple 
mentary or secondary skills. This information generally cam? 
from an interview with the employee. Unless the informatio” 
thus obtained was substantiated by the use of objective test? 
little dependence could be put upon individual claims in select 
ing persons for higher level jobs. 

6. Separation for cause. When it was necessary to remove 
an employee for cause, such action was greatly strengthen” 
by the use of objective tests of an informational or work-skills 
type if the cause was inability to do the work. 
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The Place of Testing Services in the Depot Organization 


Since testing is a staff function, serving all branches within 
the total personnel program, it cannot fully peform its func- 
tions if it is tied to any one of these branches. This has been 
conclusively demonstrated in all the experience of the Quarter- 
master and Army Service Forces Depots. When Testing was 
tied to Placement, its energies were largely absorbed and its 
Policies determined for the most part by the needs and prob- 
lems of the Placement Branch to the exclusion of Training, 
Employee Relations, and Operations. When Testing was tied 
to Training, it was found to be similarly handicapped by having 
the focus of attention placed on the needs of the Training 
Branch to the exclusion of the needs of the other branches. 
When Testing was tied to Employee Relations, it was inclined 
to take on a guidance aspect, which was not always in the best 
Interests of Placement and Training. Only when the Testing 
Unit was independent and autonomous, its chief reporting di- 
rectly to the Civilian Personnel Officer, could it avoid suffering 
from the restrictions in its activities that were invariably asso- 
ciated, in the minds of operating personnel, with the specialized 
Activities of the branch to which it had been tied. 

The desired independence for the testing functions, it was 
found, could be secured in several ways, two of which were 
Tecommended. First, Testing could be set up as a separate 
branch coordinate with Training, Placement, Employee Rela- 
tions, and Classification. Second, Testing could be set up as 
an adjunct of the office of the Chief of Civilian Personnel, in 
much the same way that Depot Control reports directly to the 

ommanding Officer. The first plan was approved and forms 
the basic pattern in most Quartermaster and Army Service 

Огсеѕ Depots. It was recognized that the particular pattern 
Of organization arrived at for any given installation should be 
determined in the final analysis by local factors. But always 
It is backed up by the recommendation from the top echelon 
that the testing activity be given the necessary independence 
to permit it to do its work unhampered by subordinating it to 


other personnel functions. 
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ORGANIZATION CHART 


Civittan Personnet Division 
Quartermaster Corps 


Кеш 


= ОА 
PLACEMENT S кам СН 
вилы Bason ми DRAM 


Another major reason for this organizational pattern wae 
the fact that the quality of personnel needed to operate ie 
effective testing program is at least on par with, if not superio" 
to, the quality of the personnel required in any other personne 
function. It is not good management to subsume one activity 
under another when the chief of the subordinate activity н 


: . Е ; ent 
classified as high as or higher than the chief of the pat 
branch. 


The Staffing of the Testing Branches 


It was soon discovered that the type and number of pe 
sonnel needed for a testing program depended on the nu™ " 
of civilian employees to be served and the variety and 3 a 
plexity of the jobs filled by the civilian employees, 25 wel he 
on the strength of the personnel program in operation 10 
particular installation. Considerable care has been give” ad 
the recruitment of trained and experienced personnel to ur 
the testing activities in the depots. As the first step 7 q- 
veloping a promising testing branch, a trained personnel e a 
nician was hired. Obviously, the successful functioning ° 
testing program will center around the qualifications an^ ps 
perience of the personnel hired for these strategic positi? jn 
The duties and responsibilities of the personnel technician pe 
charge of the Testing Section or Branch were such that if 
position always was classified at a professional level, ар 5) 
the depot was very large (5,000 to 8,000 civilian employ® 


^ 


ex 
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a fairly high professional grade was called for. These jobs 
Were set up in accordance with the specifications outlined by 
the Office of the Secretary of War. Usually the personal tech- 
nician in charge of testing had one or more professional assist- 
ants, depending again on the size and type of the depot. One 
or two clerks rounded out the office force. The following job 
description, taken from a typical depot, presents a concrete 
Picture of the type of activity involved and the type of per- 


sonnel recruited. 


Personnel Technician (Testing) P-3 

Supervision received: Works under the general administrative and technical 
supervision of the Director of Civilian Personnel, but with considerable latitude in 
Planning the details of specific assignments, and with the responsibility of carrying 
Out these assignments without detailed direction as to technique. _ 

upervision exercised: From time to time will supervise varying numbers of 
clerical and technical personnel engaged in administration, scoring, and item-an- 
alyzing of tests, rating scales, questionnaires, etc., and in the compilation of data 
Browing out of the use of,such instruments, the computation of necessary statistics, 
and the preparation of reports. Will be responsible for the training of such clerical 
Personnel in the necessary techniques, when personnel with previous experience are 


Not available. i 
, uties and responsibilities: At the X Quartermaster Depot (a Class IV installa- 

tion employing several thousand civilians in separate operating units) is responsible 
for making the preliminary selection of tests, rating scales, questionnaires and other 
evices for use in meeting specific employee personnel problems. „Such preliminary 

Selection will be subject to the approval of the Director of Civilian Personnel. Is 
Tesponsible for the administration of such tests, rating scales, questionnaires, and 
Other devices to the personnel selected, either administering the instruments per- 
Sonally or training clerical personnel to do such administration. Is responsible for 
Maintaining reasonable working conditions in the space provided for the admin- 
istration of such instruments, with respect to lighting, ventilation, freedom from 
Mterruption, etc. Is responsible for the scoring of these instruments, for the sum- 
Marization of the data so obtained, for its interpretation to the operating officials 
Who will use the information (Placement, Training, Employee Relations, Classifica- 
f reports at periodic intervals and on 

other instrument must be selected 


duties of some specific 


he Adjutant General's Office. Tf no test is avail- 
t a suitable instrument. When a test or other 
Instrument is required to cover some specific body of information, such. as m nature 
of and the regulations covering the use of the War Department Shipping Document 
it such sources as enumerated above to discover 
f none is available, construct one to fit the need. 
Is expected to construct tests, rating scales and ап апоу Mus time to time 
in connection with the selection nel for training s ae measurement of 
achievement after training. Іа all such test construction work, the procedures used 

i dpoint in line with recent developments 


im this field. In the analysis of test data or 
y be requi A E s 
fficients of various kinds, reliability and 


percentile curves, histograms, etc., and 
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to set up local percentile or standard score norms and perform other related duties 
as assigned. 

The importance of developing a working testing program 
which is firmly set on a concrete foundation of adequate рег- 
sonnel cannot be over-emphasized. 


Tests and Test Batteries 


All testing for placement purposes and a greater part of 
all other testing in the depots is done with some particular job 
in mind. Either the person tested is being considered for а 
definite opening or the adequacy of his performance in а раг" 
ticular position is being appraised. There are too many jobs 
in the Quartermaster Corps to permit the establishment of 4 
recommended battery of tests for each job separately. How- 
ever, it is possible to recommend test batteries for a series 0 
positions, such as certain series in the clerical or mechani 
fields. In a few cases specific batteries were set up for JO; 
classes or for specific jobs. This has been done for those pues 
tions which appear with the greatest frequency in the various 
depots. : 

The basic test? usually given to every employee who ! 
hired, or who is referred for testing for any reason, is the Lea?” 
ing Ability Test, which exists in two forms. This is a gener? 
verbal abilities test, omnibus type, using multiple-choice даш 
which closely resembles a general intelligence test such as t ү 
Otis type. In cases where there is a language handicap or Pe 
question of illiteracy, a non-reading intelligence test is subst 
tuted. The next most widely used test is a clerical aptitu 
test, which is given in total or in Part to all persons in cleric? 
positions. 
Other tests commonly used in the depots include the follow" 
ing: Number Speed, Typing, Shorthand, Military CorresP р 
dence, Digit Reversal, Word M eaning, Coding; Clerical Enghs” 
Battery, including tests of Abbreviation, Capitalization, бо 
pound Words, Grammar, Punctuation, Spelling, Word Divis? 
тт Typing, and Word Selection; Mechanical Aptitude "e 
Technical Aptitude Test, including the following: Mecham? 


e 
ЗАП AGO tests ar i Н in th 
tes € restricted mater Commerci used ! 
Quartermaster Civilian ils. Commercial tests are 


esting program only when AGO tests are not available. 
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Knowledge, Visual Discrimination, Space Relations, Inspection 
Speed, Technical Mathematics, Technical Reading, Figure 
Cancellation, Elementary Electricity, General Automotive In- 
formation, and Radio Information Tests. In addition, specific 
tests of knowledge on the War Department Shipping Docu- 
ments and the Vendor’s Shipping Document have been pre- 
Pared. Tests for warehouse jobs, such as packers, checkers, 
and storekeepers, have also been constructed. Most of these 
latter tests are used primarily in the training program. 
The Division of Occupational Analysis, War Manpower 
Commission, has authorized the Adjutant General to reprint 
or adapt, on a restricted basis for Army use, the tests con- 
structed for the United States Employment Service. At the 
Same time, the Oral Trade Questions have also been made 
available through official channels. In all, several scores of 
tests are available for use. 
Attempts are being made t 
Some specific jobs. Norms are 
Performance of new employees and 
following batteries are given as examp 
Prepared for specific jobs. 
Clerk-Stenographer 
Learning Ability Test 
Clerical Aptitude Test 
Typing and Shorthand Test 
Checker 
Learning Ability Test 
Clerical Aptitude Test 
Number Speed Test 

Inspector of Clothing 
Learning Ability Test 
Inspection Speed Test 
Optical Precision Stereoscope Test 
Rate of Manipulation Test 
Color Perception Test 

Fork Lift Operator 
Learning Ability Test 
Eye, Hand, Foot Coordination Test 


o set up batteries of tests for 
being gathered in terms of the 
in-service employees. The 
les of specific batteries 
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Physical Fitness Tests 
Vision 
Endurance 
Hearing 
Inspector of Shoes 
Learning Ability Test 
Inspection Speed Test 
Optical Precision Stereoscope Test 
Quartermaster Shoe Inspection Test 
Contract Negotiators 
Learning Ability Test 
Clerical Aptitude Test 
Critical Thinking Test 
Arithmetic Reasoning Test 
Personality Questionnaire 
Baler 
Learning Ability Test 
Revised Army Beta Test 
Rate of Manipulation Test 
Physical Fitness Test а іл 
These аге examples of specific test batteries assemble? е 
terms of the actual skills involved on the job. Some of t 
batteries are now in the process of validation. 


Test Records and Reports 


я Å 

It cannot be emphasized too strongly that testing 17 0 ei ) 
vice function that has no value unless the tests are у} e^ 
Hence, the system of test records and the method of interP ee 
tation is aimed at the purpose of maximum utilization 0 
results. 

Raw scores are never given to operating personnel e 5 
anyone outside of the Testing Branch. Various types 9 a ich 
are available, and their use depends upon the purpose to W e 
the test scores are to be put. The most generally used Р ad 
one in which the total group upon which the norms are a 
is divided along the base of the normal curve according hat, 
five-point scale. Each of these groups is rather easily © “of 
acterized in terms of adjective phrases, indicating 9816 


CIVILIAN TESTING IN THE QUARTERMASTER CORPS 29 


goodness in the quality, skill, or ability measured by each test. 
A test report with these descriptive phases is made out for each 
Person tested, and usually turned over to the Placement Tech- 
nician, Employee Counselor, or Training Director. In other 
words, the usual procedure is to give the operating personnel 
an interpretive comment on the test results rather than the 
test scores themselves. More detailed norms are available in 
the Testing Branch for use in cases which call for closer 
Interpretation. 
The test results (raw scores) are recorded on a Test Record 
Card which is filed in the Testing Branch. A key-sort type of 
card is the recommended record card used in most Quarter- 
master and Army Service Forces Depots. At the same time, 
test results in terms of five-step grades are entered upon the 
Employee’s Qualification Card which is maintained by the 
lacement Branch. This card is always consulted whenever 
any personnel action is contemplated. Considerable use is also 
made of percentiles as a further interpretive score. И: 
A daily log of the test results of all individuals examined is 
recorded in duplicate. One copy of the daily results is for- 
Warded to Headquarters monthly, where the results are studied 
and service-wide norms are developed. The local installation 
Uses its copies of the daily log to establish local norms. 


Validation of Tests and Establishing Critical Scores 


Attention has been given to the validation of test results 


and to the determination of critical scores. Various types of 
Criterion data, such as rating scales, quality of work output in 


terms of error scores per unit of work, and quantity of work 
Output, have been employed in these studies. Some of the 
vestigations have been discouraging in their results, especially 
in the use of rating scales. Efforts are now being made to use 
Various types of criterion data other than rating scales, to 
Show the relationship between test scores and job performance. 
tis felt by the writers that the difficulty in obtaining satis- 
factory validation data and satisfactory critical scores in many 
Cases has been due much more to the inadequacy of the cri- 


мар 
erion data than to the selected tests- 
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Quartermaster Testing Handbook 
In view of the establisment of general testing policy of the 
Quartermaster Corps and the expanding testing programs 
throughout the installations, a handbook on testing has p 
prepared. This handbook is divided into two main parts. ho 
first part discusses the role of testing in terms of the contribu- 
tion that testing can make to various phases of the depot pro 
gram, including the responsibility of the Commanding Qe 
the Civilian Personnel Officer and the Chiefs of Training, Fd 
ment, and Employee Relations. The second part discusses Es 
more technical phases and procedures of testing and is inten im 
primarily for the personnel who make up the staff of I 
Sections. The manual describes the Quartermaster L- 
Program in considerable detail and reflects the ехрепеп di 
gained in establishing Testing Sections or Branches in sev? 
installations. 
Summary 


The service-wide testing program which has been pay 
and implemented from the headquarters level in the Qu 
master Corps shows considerable promise. Ít may well pt 
the way in industrial testing, not only for other technical $ he 
ices in the Army Service Forces but to industry as well. art 
place of testing has been carefully defined as an inherent Р; Я 
of an over-all personnel program on a service-wide bate ad 
volving some 80,000 civilians employed in hundreds of diffe 
jobs. 

Briefly stated, the functions of the Testing Units as § 
within the installations under the jurisdiction of the Qu? 
master General are as follows: 


. . B 1 ше” 
1. Select appropriate batteries in terms of job red 
ments. 


ned 


et up 
геї” 


Е | s con 
2. Administer, score and interpret all tests used 1 п 
nection with placement of new employees, determinatio o 
training needs, evaluation of training, and employee rela 
activities. ing 
. at 
3. Conduct testing surveys at the request of oper?" jy 


officials, and provide suitable reports of results through Р!0Р 
channels. 
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4. Conduct the necessary research to establish local norms 
and to determine the validity of experimental tests. 

5. Construct new tests as required. 

6. Maintain necessary records. 

7. Coordinate testing with other personnel activities. 

Testing is only one aspect of the Civilian Personnel pro- 
gram in the Quartermaster Corps. But it is an important aspect. 
It provides valuable information concerning the employee's 
abilities, aptitudes and skills, obtained in a relatively short 
Period of time. Tests, wisely used and with due consideration 
to their limitations, enable the immediate location of the more 
apt workers and the more trainable employees. Tests are 
making an important and vital contribution to the war effort 
by insuring the maximum utilization of manpower. It can 
truly be said *Tests have gone to war on the civilian as well 


as the military front.” 


TESTING BY MEANS OF FILM SLIDES WITH 
SYNCHRONIZED RECORDED SOUND 


HERBERT A. THELEN 
The University of Chicago 
1. Preliminary Considerations 


FuNpAMENTALLY the method of evaluation is to put the 


st : : й : ; à 
udent into situations likely to result in experiences engen- 


deri e . ex 
ring overt responses which can be used for valid prediction 
goals of education. 


of be A : 
qe iene assumed to constitute the 
€ implementation of this method is seen to require a clear 


i T of educational objectives, the setting up of con- 
ed patterns of stimuli appropriate to the level of maturity 
and other characteristics of the students, the description of 
Fi responses of each student, and the generalization of the 
арбаша of these responses into predictions of types of 
i aa in a wide range of similar situations, and finally, the 

uation of the degree of appropriateness of the responses 
ОЁ each student as compared with those of his classmates. 
We For the purposes of this discussion of a new type of test, 

„shall postulate that by a “test?” we mean an instrument 
Which measures status of a student relative to certain objec- 
ves and relative to a group of students tested at the same 
time. We shall further assume that the instrument is con- 
Structed after the specific objectives have been described 
°Perationally, and that the test situations have been formu- 
ated to elicit behaviors to be appraised with due regard for 
арргоргіасепеѕѕ of subject-matter content, level of maturity, 
sai type of problem tension engendering the observed re- 
POnses, These assumptions are required in order that dif- 
‘rent media of tests may be compared. Р 

A test is of little value if the results of testing cannot be 
Mterpreted for some clearly stated purpose. Probably the 

38 
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major factors in the interpretation of a test to which the above 
postulates apply are: (1) adequacy of available description of 
the testing situations, (2) degree of insight of the interpreter 
into the psychology of learning and behavior, and (3) knowl- 
edge of the culture, maturity, tool skills, familiarity with situ- 
ations similar to those in the test, and other relevant attributes 
of the students tested; for these three factors determine the 
validity? of description of the behaviors presumed to account 
for the responses checked in each test item. Certain aspects 
of these factors may now be considered in detail. 
_ 4. The Testing Situation. At the operational level, а tes 
is generally a piece of paper with writing on it. For most 
students it presents the dominant stimuli in a total situatio” 
which includes the test administrator, a group of students, 
the physical environment, and the temporal place of this 819 
uation with respect to an undefined sequence of experience 
situations, both prior and subsequent, each of which has €". 
gendered or anticipated a more or less relevant experience k 
the student. The only factors which may be clearly describe 
in the testing situation relate to the test itself, and, with 
lesser comprehension, to the procedure for administering © 
test. If the test actually does elicit the full attention of us 
student, then the remaining physical factors are assume 
be of negligible importance in dete е 
factors of relevant previous experience are assumed to Ре a 
major cause of differences among the scores of the student 
It follows that to describe the testing situation adequate d 
one must know: the effectiveness of motivation of each stude? 
s: exact procedure for giving instructions (and makin£ ei 
‚> EM есиги d the relevant physical sar M 
inistration of the test, and, finally», 


r s 
nature of the test items (language used, problem-ten? Кі 
ume attitudes implied, slogans employed, and the us £ 
ince the extent of the m 


measured di otivation of a student cannot 
a sure directly, and can be inferred only in the gas 
me tests giving a pattern of scores, it is customary to 455 


vs nd the 
rmining the score, а 


1 Define T 
annor a as amount of correspondence between predicted and observ” 
entire population of problems sampled by the test. 
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that all students are either equally or maximally motivated. 
Since the extent of a student’s understanding of the directions 
for taking the test is not measured apart from his achievement 
on following the directions, it is assumed either that the under- 
Standing of directions is perfect or else that ability to under- 
Stand directions is part of the achievement being appraised; 
the latter is probably the sounder assumption. The usual 
assumption about physical and social conditions is that good 
lighting, adequate ventilation, and an empty seat between 
adjacent students are about the only relevant factors. Since 
а test should not be given if it is inappropriate for the group 
being tested or if it does not measure the desired objectives, 
it follows that use of a test assumes that it will be valid under 
the conditions in which it is used. This assumption can be 
Justified only by detailed analysis of the individual items with 
reference to such factors as those suggested above. The neces- 
Sity for making these or similar assumptions confronts the 
Interpreter of any test. The justification for these assump- 
tions may, however, vary from one test situation to another. 
t would be desirable to make these assumptions as valid as 
Possible. 
B. "Verbal" versus “Real” Situations. If a test is used 
Merely as a hurdle to be cleared for the sake of a certificate, 
then, within wide limits, the nature of its items makes little 
ifference so long as the distribution of scores covers a wide 
Tange and the students opined to be “best” receive the highest 
Scores, and the students believed to be “poorest” receive the 
West scores, Such a test used in this way does not require 
discussion here because the preliminary postulates do not apply 
to it. The statement that “specific objectives have been de- 
Scribed operationally” means that the major, generally-stated 
Objectives have been analyzed and broken down into a large 
number of distinguishable types ofaction. Each of these types 


as its own properties and relationships with other types; these 
relationships constitute the rationalization of the general objec- 
tive. In a course for which such an instrument is appropriate 


as an aid to evaluation, there has been some effort to teach 
the Students the types of behavior, the criteria by which the 
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most appropriate type may be selected in a given situation, 
the rationalization of the general objective, and the scope of 
applicability (range of situations and purposes) of the objec- 
tive. These objectives are generally understood to refer to а 
wide range of everyday experience; but their measurement is 
essayed through symbolic verbal experience. Under these 
conditions it is not difficult to come to the conclusion that all 
mental processes come under the heading of “reading compre 
hension" and that therefore the major task of schools is 10 
teach students to read. 

Situations as conveyed by paper-and-pencil test differ from 
the situations encountered in everyday life in some very im- 
portant respects, and these also present the test interpreter 
with the need for making a number of assumptions whose valid- 
ity is generally not easily appraised. Some of the factors 
involved are: 

(1) Ambiguity. Through verbal symbols standing for 
objects one attempts to engender the same response that the 
objects themselves would produce. Since object-names stan 
for classes rather than for individuals, a number of qualifyin8 
adjectives (or stated properties) must be given to specify the 
individual. But the adjectives themselves are usually 25807 
ciated with outstanding types of objects to which the adjec 
tives best apply (within the experience of the reader), 50 that 
the reader is confronted with the task of visualizing a specific 
pattern of aspects (which in turn can never completely p 
produce an object) from what amounts to a series of gener? p 
zations about varieties of types of objects. It would be quite 


Seer Jt à o 
surprising if such a description meant the same thing 10 
different students! 

tern 


(2) Semantic misrepresentation. The particular pat he 
t 


of aspects through which the tester attempts to convey 
object to the reader's experience may or may not coincide Wt 
the pattern of aspects by which the reader symbolizes or wou 
symbolize the object. 

(3) Incompleteness of pattern of stimuli. All the sense? 
together enter into the experiencing of an object or situation, 
Presumably the number of possible kinds of response won 
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depend partly upon the complexity of the pattern of stimuli 
(in this case represented symbolically ). There is no adequate 
symbolic notation to even represent odors, tastes, or surfaces; 


and therefore aspects ordinarily obtained through these types 


of perception cannot be given. This is one kind of cultural 


limitation imposed upon test items. It may be argued that 
this limitation is unimportant because we make little use of 
Perceptions from these senses (since they cannot be repre- 
sented—a vicious circle), yet there is undoubted depreciation 
of the validity of the vicarious experience engendered within 
these limitations. 

(4) Non-simultaneous presentation of the pattern of as- 
Pects. In reading, images are built up piecemeal, and with 
extremely attenuated impact. Furthermore, the order of pres- 
entation of the components from which the pattern is formed 
is fixed. All three of these factors are artificial and may be 
expected to give the test situation different meaning from that 
of the corresponding “real” situation. 

(5) Selection of aspects. Any symbolic system of repre- 
Sentation proceeds by selection of relevant aspects. '[his 
focuses or fails to focus attention upon details whose impor- 
tance is thus magnified or diminished out of proportion to the 
other details in the situation. 

The above theory is present 
Symbolic representation some o 
With paper-and-pencil tests. In 
these tests present artificial situ 

inds of response is limited, and 
of verbal symbols is an importan 


unknown degree the nonreading abut! 1 
Use of such tests has led to emphasis upon learning of self- 


Contained relationships among symbols rather than upon phe- 
Nomenal aspects represented by the symbols—students are 
» 


EE 
taught “maps” rather than "territori | 
It seems reasonable to suppose that instruments dealing 


With experience solely at the verbal symbolic level тау; never- 
theless, be of some use in evaluating abilities which are defined 
In terms of behaviors guided largely by verbal maxims or 


ed to rationalize by means of 
f the experienced difficulties 
general, it may be said that 
ations to which the range of 
that facility in manipulation 
t factor which masks to some 
bilities to be measured. The 
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conventions. Although the situations eliciting the behavior 
are subject to the difficulties outlined above, the behaviors in 
these situations may be largely explained as manipulations 
with verbal criteria. О Е 
The behaviors encompassed under the generic title of crit- 
ical thinking,” because of their high symbolic loading, can 
be appraised much more satisfactorily than can “attitudes. 
Thus judgments in terms of stated or unstated criteria, or in 
terms of logical rules or scientific methodology may be elicited 
with paper-and-pencil items. The major discrepancy may 
here be found in that the behavior starts after verbalization 
with the verbal items, bnt before verbalization with the non- 
verbal items. The apffraisal is then limited to a part of a 
process rather than to a whole process. If this limitation 1S 
recognized, however, interpretation may be apparently sound. 
One obvious way to overcome the difficulties inherent in 
a verbal presentation of situations is to place the student in a 
more or less controlled “real” situation and then observe his 
behavior. This sort of technique usually involves individual 
testing of each student by a specially trained observer; the 
results may be expected to be less reliable but more valid. 
Even here, however, the testing situation differs from the 
population of situations about which we wish to make predi- 
cations in that the problems are formulated or at least sug- 
gested by the observer; that is, the tension resulting in prob- 
lem-solving behavior of the testee is stimulated by the ob- 
server rather than by the configuration of naturally occurring 
elements within the situation itself. This technique is ad- 
mittedly cumbersome and expensive. ; 
‘The present investigator has become interssted in the poss! 
bilities of the sound-shde medium Yor reducing the Ма 
verbal symbolism and increasing the participation of student? 
in testing situations. The test so far constructed will be 9 
scribed and then tentatively evaluated by means of some © 
the concepts stated above. 


2. Description of the Test i 
ibe 
A, Nature of the Instrument. The test to be a 
simultaneously presents a controlled pattern of stimuli apP 
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mating “real” situations. To this extent it resembles the con- 

trolled observation technique more than that of the paper-and- 

Pencil test. The overt responses of the students are limited 

to recorded judgments, opinions, and analysis, and to this ex- 

tent the test is akin to a paper-and-pencil test rather than an 
observational type of evaluation. 

wi rs particular instrument to be described is concerned 

ith the area of behaviors generally referred to as “ability to 

| apply principles.” Тһе principles are taken from among those 

Ordinarily studied in the fifth-grade physical science units at 

the University of Chicago Laboratory School. 

The test consists of a film strip, а recorded transcription, 

ч Sur sheets for the students. "The test is given in a 

idarkened room; the light is adequate for students to follow 

the answer sheet and dim enough for the projected pictures to 

time ] The film strip is projected one frame ata 

» and the pictures are changed at a signal recorded in the 


transcription. The recording provides some narration for each 
ts, and directions for marking 
Sixteen-inch records are used; 
and run for seventeen minutes. 
e to five pictures per problem, 
pewritten titles. Possible 


е clearly seen. 


iSi ui, authentic sound effec 
Teer on the answer sheet. 
are played at 33 1/3 r.p.m. 

he film slide strip presents on 


T also contains photographed ty 
nswers are presented as depicted right and wrong Ways of 


ing certain jobs, written explanations or principles, depicted 
Members of analogies, depicted illustrations of operation of 
Principles, and other devices deemed appropriate to the specific 


objective being tested, А few of the problems WW МАМ 


wi 
Titten Statements, 


The Present test ; 

Operator was given in Gr d 

а Stopped ades V. 

E per E ETE ОТЫ ы 
"he {гайэ AURREI ТЕР wi ate А 4 


The 2 
Pauses prov, ір cription Were арек 


Correct fi к 
ia the tenth-grade students, hut had to 6e lengthened 
tenth ue ers. The test required thirty-three minutes in th 
E e се about forty-eight minutes in the fifth. ша 
Diners: Тет Tiems and Objectives. The test 
Steen problems focused on ten specific stared еннен 
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the area of application of principles in elementary science. 
The following brief description of the objectives and items may 
suggest sorts of possibilities for testing with this medium: 
Objective I. To recognize a practical (unstudied ) appli- 
cation of a principle studied in class. 


Problem 1: A ruler clamped in a vise is plucked, and the 
sound is heard. Then a shorter ruler is plucked, and the 
sound is heard. The students are asked to consider three de- 
picted ways of getting different notes from a violin: by turn- 
ing a peg, playing open strings, or playing up and down the 
scale. For each of these three situations they record that the 
notes are different for the same reason that the notes from 
the ruler are different, that the notes are different for some 
reason other than that shown with the ruler, or that there 1S 
not enough evidence to decide between the alternatives.” 


Objective П. To arrange events in a temporal sequence uad 
cording to a developmental principle. 


Problem 2: Three pictures designated А, В, and C show 4 
man using a lathe in the construction of a gadget, using me- 
chanical drawing instruments in designing the gadget, and hav- 
ing an inspiration for the gadget. The student writes the 
letters A, B, and C, in the order indicating the sequence aS Р 


thinks it really occurred. 
ple) the 


Objective III. To recognize (from a studied princi 
best technique for solving a simple problem. 


Problem 3: Situation: Grease in a skillet has caught pe 
Student selects the better depicted method of putting out p г 
fire. Choices: Clapping a lid on the skillet; running wate 
into the skillet. T 
Problem 4: Situation: Balancing of large weight and amo 
weights hung from a differential pulley. Choices: же 
weight hung from smaller diameter; large weight hung fro 

larger diameter. 5-6 
Problem 5: Situation: Location of tray for fastest freezing "n 
the cooling unit of a refrigerator. Choices: Lower right-han 

corner of freezing unit; middle top shelf of freezing unit- 
Problem 6: Situation: Position of head to hear a tuning face 
loudest. Choices: One ear directed toward the fork; a 

turned toward the fork. 


? This is an analogy between a studied laboratory situation а 
practical experience. The converse objective, recognition of a 
which operates on the same principle as a familiar practical proc 
not tested. 


evices 


je 
nd an поз ор 
laboratory "25 


| 
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In these problems the student selects the better choice, 
selects both choices as being satisfactory, or rejects both 
choices. 

Objective IV. To recognize situations illustrating the opera- 
tion of a stated principle. 


Problem 7: Stated principle: A smaller force can overcome a 
larger force provided it moves farther and faster than the 
larger force. Situations: Boy lifting handles of a wheelbar- 
row; jackscrew raising a heavy load; single fixed pulley sus- 
pending two equal weights. : 
Problem 8: Stated principle: Sound is produced by the vi- 
brations of objects. Situations: Block of wood hit with 
hammer; water poured from pitcher to glass; whistle being 
blown.? 


In these problems the student rates each situation as illus- 
trating the principle, not illustrating the principle, or as in- 
sufficiently described for a decision to be reached. А 
Objective V. To identify a simple familiar mechanism or 


Process, 


Problem 9: Find the wedge. Situations: Driving a nail; 
Sawing wood; pulling a cart up an inclined plane. 


Problem 10: Find the sound being reflected. Situations: 
оу shouting “around а corner"; man shouting in a large 
empty room; hammer in piano striking a string. 


In these problems the student rates each situation as de- 
Picting the mechanism or process, as not depicting the mech- 
anism or process, or as insufficiently described for a decision 


to be reached. T 
bjective VI, To compare predicted (from familiar, unstated 


Principle) results with observed results in simple laboratory 
Situations, 


Problem 11: What is wrong with this picture? Situation: 
hina dish is shown being heated by the luminous flame of a 
Bunsen burner, Then the flame is turned off, and the dish 


15 seen to be clean. 
Problem 12: What is wrong with this picture? Situation: 
imetallic bar is shown to be uncurved in a hot flame, and 


—<urved after cooling. 


Sui op his problem may also be. regarded as a sort of logical tautology, since any 
Were prota production must "illustrate" the principle. The tenth-grade students 
tween ably sensitive to this aspect, whereas the lower grades distinguished be- 

е whistle (a musical instrument) and the other two sources (noisemakers), 


i 
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In these problems the student rates the second picture as 
depicting a correct result, or else explains briefly what 28 
incorrect. 

Objective VII. To rate statements of principle or fact as 
useful in explaining depicted phenomena. 


Problem 13: Phenomenon: Whispering is transmitted by а 
garden hose. Statements: “Some solids transmit sound better 
than air does.” “Sound is reflected by surfaces.” “Whisper- 
ing is higher pitched than talking.” “Sound travels outwar 
in all directions through a gas.” “Sound is partly absorbe 
at surfaces.” 


Problem 14: Phenomenon: Pitch of a tuning fork is the same 
whether hit hard or softly, whether held in the air or mounte 

on a resonance box. Statements: “Rate of vibration of an 
object depends upon its size.” “Loudness of a sound depends 
upon how much the source vibrates.” "Sound is producec 
by vibrating objects.” “The number of overtones in а sounc 
depends upon the construction of the source.” “The rate 9 
vibration of a string depends upon its tension.” 


Problem 15: Phenomenon: Man sitting on stepladder in 2 
room is warmer than he would be on the floor. Statements 
“The floor conducts heat more rapidly than the се! ings 
‘Heat rises because it is lighter than cold.” “Less депле 
objects float in a more dense liquid or gas.” “А certain weight 
of air occupies more space when it is hot than when it 15 CO! 3 
“Heat is due to moving molecules and therefore hot thing 
move more rapidly than cold things.” 


In these problems the statements following the descriptio? 
of the phenomenon are flashed on the screen one at 2 we 
The student rates each statement as being helpful in the © 
planation, or as not being helpful in the explanation. de 
Objective VIII. To identify an incorrect postulate in the 
picted solution of a problem. 


Problem. 16: 'To tune a viola. Depicted solution: The viola 
is tuned aurally, and the position of each peg marked by теат” 
of a scratch. From then on, the instrument is tuned by trc 
ing the peg to line up the index and scratch. The experime 

is tried, and the viola heard to be out of tune. m 
Problem. 17: 'To keep cool on a hot day. Depicted soluti; 
A boy is shown putting on successively a sweater, a coat; s 


nc" 
4 i cie H „h rele? pe _ 
The criteria or conventions of explanation tested are concerned with p nd tb 


ени of statement, closeness of analogy, description of mechanis™ 
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a blanket. In dialogue with the narrator he explains that 
this will keep the hot air from reaching him, but admits that 
he is still hot. 


In these problems the student writes a brief criticism of 
the solution shown. 
Objective IX. To select the best stated prediction in a prac- 
tical unstudied situation. 


Problem 18: Situation: Taking the nut off a large bolt. A 
| pipe has been slipped over the wrench handle, and a man is 
pulling on the pipe. Stated predictions: “The pipe will bend." 
“The nut will come off.” “The bolt will be twisted in two.” 


“Nothing will happen.” 


The student indicates the prediction he thinks most tenable. 
Objective X. To select the most appropriate opinion (verbal 
expression of attitude) about the desirability or undesirability 
of a depicted situation. 


Problem 19: Situation: A pile of trash and old newspapers 

in the corner of a “dark, warm basement” is shown. Four 

Opinions expressing different degrees of alarm over possible 

anger are given. 

н The student selects the opinion which he most nearly agrees 
With, 
. Problem 3 is depicted in full. The technique of testing 
Includes the following steps: 
. (1) Presentation of a title or statement designed to gain 
Interest, to indicate the general nature of the task, and to 
mark the beginning of a new problem. The title is read by 
the narrator while it is on the screen. t 
М (2) Description and depiction of the problem situation. 

Pictures are arranged in sequence, and, together with the nar- 
"ation, tell a simple story. 

(3) Presentation of answers from which to select. These 
Тау be depicted ways of doing things, verbal statements 
(which may or may not be read by the narrator, depending 
Upon the objective) of explanation or prediction, and the like. 
П some problems the student is asked to write in his explana- 
Чоп or criticism (short-answer essay). 


m. 


e^. | 
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(4) Giving of directions by the narrator for answering 


the item. 

(5) Allowance of sufficient time for all students to mark 
the answer sheet. 

Steps 2 and 3 may occur simultaneously. Step 4 may 
precede 3, or even 2. 


Problem 3 -M 


y 
к e (ша Recording (narration, directions, sound effects) Р 
and pictures) А 

Narrator: What is the right way to put out te 


fire in a skillet? 


What is the right way 
to put out the fire 
in a skillet? 


Sounp Errzcr: Bell signal. 


. Я : ens 
Narrator: This sort of thing sometimes hapP 
when we heat a skillet with grease in 1t. 


How shall we put out the fire? — 
(See Plate 1) 


Sounn Errect: Bell signal. 


З ; к : way 
Narrator: Is covering it with a lid a good 


to do it? 


(Pause) à 


(See Plate 2) 


Sounp Errect: Bell signal. р 
te 
Narrator: Or would it be better to pour "^ 
on it? 
(Pause) y 
In answer space 5, write A pon hut way E 
the right way to put out the fire, Wr Im 
(See Plate 3) second way Sa M right way to put аш етан 
write both А and B if both were correct, 0 is 
a dash if neither way was correct. (Йаш hat 
ping transcription if necessary, until all of © 
marked the answer space.) 
7° 


‚С. Results of Testing. By means of this instrument Mv 
ing the sound-slide medium, it was desired to explore 2 ашла 
of possible types of items and situations. The brief samt igh 


of the various objectives precludes expectation of any cbe 
degree of reliability with respect to single objectives. pur 


was 
t 


№ 
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more, if the objectives actually are different types of behaviors, 
then the test as a whole would not be expected to have high 
internal reliability. On the other hand, the test could be 
Considered valid if predictions guided by an acceptable theory 
of learning and based upon knowledge of the learning experi- 
ences of the students were borne out by the test results. It 
was believed reasonable to suppose that this test, presenting 
the stimuli simultaneously and with a minimum of verbal 
Symbolic representation, should have high face validity. To 
test this hypothesis a series of predictions about the results 
of testing were set up and studied. The predictions were: 

(1) Accuracy on the test as a whole will increase with 
the grade level of the students. 
. (2) Increase in the appropriateness of responses in par- 
ticular items will spurt between the grade levels above and 
elow which the relevant principles were studied. 

(3)" There should be some evidence of increasing maturity 
Sf thought discernible in the’ pattern of responses as grade 
evel increases. While such patterns have not yet been de- 
Scribed adequately, there should be agreement with such frag- 
ments of information as are now available. 

he results bearing upon these three predictions are: 

r a Median scores: fifth grade 16.8, seventh grade 19.5, 
Bath grade 21.0, tenth grade 23.5.2 > ; 
of ) The placement of principles in the science curriculum 
$ the Laboratory School has been relatively constant for 
мы years, but the courses have been taught by several 
ed €rs, some of whom are no longer in the school. Knowl- 
‚Бе of the learning experiences of the students was consistent 
Observed spurts in accuracy relative to all the items for 
at h such knowledge was available.” In other words, the 

results reflected the learning experiences faithfully so far 


a 
ey could be described. 


$ 5 . . 
та sThe large influx of students new to the school in the ninth grade may result 
No atte What low median for the tenth grade as compared with the other grades, 
Teasg ыр? was made to match the samples of students because there is no good 
апу ini SUPPOse that the students in Grades V, VII, and УШ are different in 
в ApPropriate dimension. 
Out half the responses. 


it 
Whic 
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(3) The evidence concerning the changes in pattern of 
accuracy from grade to grade is meager and subjective. Re- 
garded as empirical finding, it would be worthless, but its con- 
sonance with parts of the pattern of anticipations increases 
the validity of the instrument by some unknown amount. 

(a) The test key calls for two responses that “more in- 
formation is needed for a decision.” In Grade V the 
accuracy was approximately that expected by chance; 
there was a decrease in accuracy up through Grade. 
This is in agreement with the common observation 
from Interpretation of Data Tests that “tendency t? 
go beyond the data” increases with grade level (in 
the absence of special training). 

(b) Accuracy of rejection of the irrelevant reasons in 
lems 13, 14, and 15 increased markedly between the 
eighth and tenth grades, but it cannot be shown that 
subject matter which may have been presented during 
the ninth grade does not account for the gains. 

(c) The decline in accuracy with principles kno 
have been studied in the fifth or sixth grades an 
reviewed subsequently appeared to depend upon р 
directness of applicability of the principle as Jearne™ 


prob- 


wn t? 
d not 


3. Criticism and Evaluation of the Medium 


ж y 8... А А » ove 
The discussion under *Preliminary Considerations ab 


provides several criteria which may be used in evaluating ? 
criticizing the type of test here dealt with. 
i Two facts give us some assurance that the testing si 
is controlled and therefore definable to a high degree: he 
instructions for taking the test are recorded, and this fixes E. 
factor which is usually the most variable in the administr? Ko 
of tests. (2) The test holds the interest of the student? fact : 
motivates them to work intensively. This is stated aS ĉ ir, 
as a result of discussions of the test with the classes raking s 
and as a result of observations of behavior of the stud? 
while taking it. i 
The “realness” of the test situations is greater than vot 
paper-and-pencil tests. Consequently it should enable | 


ion 
џай0 
t ll 
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valid predictions as to the behavior of students in similar “real” 
Situations, and this type of prediction is assumed to be the 
Most legitimate purpose of achievement testing. The use of 
Motion pictures for depicting some of the situations involving 
changes along the time dimension would presumably increase 
the “realness” further (as would also the use of stereoscopic 
Pictures and color. Whether this increase would justify the 
Increased expenditure of effort in making the test is not known; 
careful analysis of the objectives and situations would enable 
Опе to set up hypotheses). 

The sound-slide medium very much minimizes the cus- 
tomary use of verbal symbols in conveying the situations; this 
should make possible the evaluation in the lower grade levels 
9f some behaviors hitherto not readily available for testing. 
(An illustration is the identification of assumptions in prob- 
€m 16.) The minimization of reading comprehension as a 
Prime factor in determining the student's responses should also 
make possible the testing of many objectives more directly. 

The more complete presentation of situations by picture 
and sound means that the pattern of stimuli comes closer to 
actual experience. Coupled with the advantages listed above 
may be an increased difficulty of “focusing” items so that the 
Student does not respond unduly to irrelevant stimuli. In 
other words, the more completely the situation is conveyed, the 
Breater the number of possible types of response, and care 
(USt therefore be exercised in stating the question unam- 
.ÉüOusly so as to elicit the type of response which is most 
Informative in the evaluation of the objective to be appraised. 


4. Plans and Suggested Possibilities 


н Other factors being equal, the more adequately a situation 
Presented, the more valid the response. It séems reasonable 


o > 3 3 
ali Suppose that this medium may have interesting potenti- 
Чез for the appraisal of attitudes. Instead of stating an 


aoe as to preference in verbalized general situations, a 
^ ent might be asked to criticize a depicted course of action, 
: 00 choose among several depicted solutions to a problem 


Mvo]y; 3 
olving a conflict in values. Instead of having to select the 
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relevant aspects of a situation for the student (as one must 


do in verbal presentation), it would be possible to present 
subtle but crucial factors disguised in situations. In this 
event the responses of the student might be governed more 
by the values he lives by and less by the slogans he has learned. 

The science department of the Laboratory School is work 
ing as a group on the construction of a sound-slide test 10 
appraise ability to form reasonable conclusions. A variety 9 
situations in and out of science will be used in an effort to 
find out whether this ability can be described as an entity apart 
from associated learnings of subject matter. The identifica- 
tion of a number of such abilities plus the development 9 
adequate means of appraisal would make possible some me 
nificant research on teaching methods, and might well lea 
to a complete reorganization of the content of elementary 
science courses. 

5. Summary 

h synchron” 
s describe® 


f ability 


A new type of test making use of pictures wit 
ized narrative, sound effects, and instructions i 
The use of such a test for appraising some aspects 0 
to apply elementary principles in science 15 explored. 

Advantages claimed for the sound-slide test are: 
formity of administration of the test from group t9 
(2) high motivation of the students, (3) minimization sec" 
verbal element with increased validity of testing some ^ a 
tives, (4) possibility of appraisal of some fairly sophistic? 
objectives at low-grade levels. 


(1) uni- 
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ANALYSIS OF THE TERMAN-McNEMAR TESTS 
OF MENTAL ABILITY 


F. T. TYLER 
The University of British Columbia, Vancouver, B. C. 

The Terman Group Test of Mental Ability was probably 
Опе of the most commonly-used group intelligence tests over 
= Period 1921-1941 (8, p. 33). It is likely, therefore, that 
s © revision of this test will be of considerable interest to 
chool officials, Analysis of the revised form by Terman and 
€'Nemar should give valuable information to supplement the 
; anual of directions: The purpose of this paper is to present 

© results of such an analysis. 


m 


The Subjects 


мые Subjects were students in the junior high school at 
iter o British Columbia, where it is the practice to admin- 
agi ae intelligence tests in grades 7 and 9. Approxi- 
emb У 100 students took Form D of the TMcN* tests in Sep- 
КА ®t, 1942; forty-nine of these had previously taken the 
P tests in 1940 in grade 7. The TG test was administered 
l of the grade 9 pupils in October, 1942. Form C of the 
rr test was given in February, 1943, to 88 of the grade 9 
fan for whom Form D scores were already available. 
in Tar between I.Q.’s on the various tests are shown 
«ы average TG I.Q. in grade 8 in the Vancouver, B. C. 
Brade System was 106 in 1940 (12, p. 106), rising to 115 in 
Btade к It is likely, therefore, that the average LQ. in 
15 about 108 ог 109. The subjects used in the present 
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study appear to constitute a typical sample, although possibly 
slightly above average. It should be noted from the table 
that the average DIQ of 88 cases is three or four points below — — 
the average for 49 cases. It seems reasonable to expect, there- | 
fore, that the average КА and TG I.Q.’s for the whole 88 cases | 
would be somewhat lower than those for only 49 students, 50 
that the average TG LQ. approximates that found in the 
Vancouver schools. 1 
Despite the apparent differences between RIQ's and DIQ's 
the correlations between them are very high, being .92 and . 
for Forms C and D, respectively, practically identical with the 
relationship given in the manual of directions, namely, -92 for 
Form C. As the authors state: “From these data it will be 


TABLE 1 
Means and Standard Deviations of sic on Vac Tests p. 
Form D Form D Form e. 
KA TG — Du 
RIQ DIQ ко DIO RIQ 
Duc 140 1942 1з 19 19 10 юз 1948 
N 49 49 49 49 88 88 88 110 
M 109 113 122 113 115 109 116 13.08 
с 11.30 10.65 20.62 11.65 19.73 13.40 18.02 Y 
EN 0 
с Manual of directions 29.10 171 
very 


seen that the rank order of deviation and ratio 1.0.5 15 
nearly identical, but that the magnitude of the 1.0.8 will va 
in increasing amount as one moves away from the mean’ 
р. 10). This also accounts for the fact that the mea? ^ 
of the present sample is larger than the mean DIQ. The b. 
ference in value between a student's two 1.Q.’s need not conc 4 
the teacher if he understands that a difference is to be exper ig 
because of the differences in the standard deviations of the d 
types of scores. The manual of directions might һау 
more explicit on this point for the benefit of those | a 
who are relatively unfamiliar with the meaning of 2 sta 
deviation. The authors recommend the use of the DIN a 
the RIQ will be used in many schools because teac e 
more familiar with its definition. 


Д 
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Findings 
l. Correlations between Various 1.025. 
Table 2 shows the correlations between the various 1.0.8 
obtained in the present study. í 
TABLE 2 


Correlations between 1.Q.'s 


Variables oe, N P 
2 55 .79 
2 68 ‚78 
2 71 77 
2 71 S84 


. 
А ; = -—-— 
2 Correlations are statistically significant. | 

m D was used in this part of the analysis. 


ЖЗ Correlations between 1.Q.’s are very similar, none of 

With th rences being significant. The TMcN 1.Q.’s agree as well 

сое с; е other tests as the others agree with one another. The 

intell; lents are similar to those usually reported between group 
Bence tests. 


р; 
"ficulty of the Tests. 
€ans and standard deviations of various scores are shown 


in Table 3. 


TABLE 3 
Set. Means and Sigmas of Scores on Tests and Subtests 
Form D 
Subtest Form C orm 
M с M с 
1 
18.0 3.61 169 3.21 
2 116 $18 119 180 
3 16.3 5.60 151 424 
+ 179 2.60 172 321 
3 173 4.05 163 3.85 
p 128 4.02 118 107 
Т, 9.9 1.85 8.7 2.17 
A 103.5 18.80 978 19.70 
175 244 169 2.51 
116 11.79 14 1231 


e E 
n di ue be seen that the two forms are distinctly comparable 
ulty, ang variability. The average percentages of all 
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items passed are 64 and 59 for Forms C and D, respectively. 
The authors report average difficulty values of about 56 per 
cent for grades 7, 9, and 11. The higher average per cent 
success on Form C than on Form D for the present sample 15 
explainable in terms of growth, with possibly some practice 
effect. и 

The subtests vary considerably in mean difficulty and vari- 
ability, subtests two and six being significantly more difficult 
than the others. 


3. Item Difficulty. 


Form D was analyzed to determine the range of difficulty 
values of each item. These are shown in Table 4. 


TABLE 4 
Range of Percentage Success by Items — " 
= ; betwee” 
Range of per cent Per cent of items Н 
Бона success 40 and 59 per Сей 
1 18-98 16 
2 4-92 24 
3 16-98 28 
4 12-100 16 
5 8-98 4 
: 14-95 16 
59-96 m. me 


в | апё5 
With the exception of test 7, the range of success мая 


from a low to a high percentage in each subtest, а situat 
usually associated with maximum reliability (4, P- 3 
the other hand Symonds (10) and T. G. Thurstone (15) ultY 
shown that a test consisting of items of fifty per cent di V 
value measure an individual most accurately. Compara ef 
few of the items on this test fall within the range 40 10 
cent difficulty value. owe! 
The authors believe that the test is essentially 2 Pres” 
test, i.e., that the items have been arranged within eac 9° pis 
in increasing order of difficulty with ample time limits: Quo 
claim was appraised by computing the rank order corre "os 
between obtained order and test order in the subtests ? 
D. These are given in Table 5. 
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TABLE 5 
Rho between Obtained and Test Order of Difficulty 


Subtest rho* 


MIO Un de U N e 
o 


. . H . 
All values of inferred r are significant. 


The values of these correlations indicate that essentially 
the items are arranged in order of difficulty. Despite this, 
item analysis indicates that for the Canadian sample some 
items are very seriously misplaced. These results may be 
compared with those of Hovland and Wonderlic, who report 
rank order correlations between test order and obtained order 
of Аб to .75 in various forms of the Otis Self-Administering 
Test, Advanced Form (6). 

‚ There is no definite way of knowing which items a student 
tried, but for purposes of this analysis it was assumed that a 
Student attempted all items down to the last one he marked. 
| able 6 shows the percentages of students who marked the 

ANE item in each subtest, i.e., the percentages who attempted 


al items, 
TABLE 6 
Percentages of Students Attempting All Items 


Subtest % 


ION Un di UGN = 
ч 
n 


Evidently the test is essentially a power test, since such 


lar : ^ 
mb. Dumbers of subjects were able to try all items in each 
test 


Suitability of the Test at the Grade 9 Level. 
he fact that about 60 per cent of all items were success- 
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fully passed suggests that the test might be too easy at the 
grade 9 level. Table 7 shows the percentages of subjects who 
obtained mental ages of 19 and over, and 20 and over, ОП 
each form. 

Since the tests fail to discriminate between the mental ages 
of such a large percentage of students, and because so many 
earned the maximum mental age, it seems reasonable to com 
clude that the tests are too easy at the grade 9 level, and pos 
sibly even for bright students in grade 8. This test apparent у 
suffers from the same weakness as did the TG test: “As Теш! 
points out, a child capable of earning a score of 180 or bette 
is under a handicap” (1, p. 157). A 12-year-old student may 


TABLE 7 
Percentages of Students with M.A/s of 19 and 20 = 
Form С Form D = 
M.A. E EE 
No. » No. а 
19 and over ....... 32 36 25 5 
20 and over ....... 22 25 14 M 


an 
€ 


earn a DIQ of 161, whereas the highest DIO obtainable 2, 
18-year-old is 138 (11, Table 3). DIQ's are probably " 
satisfactory than are RIQ's, but the test appears to be 100 an 
for students above grade 8. This should be verifie 
analysis of the test results of grade 11 students. 


5. Reliability of Tests and Subtests. 
Reliability coefficients were determined by corr? 
scores on the equivalent forms. | du 
: The inter-form reliabilities of the subtests vary rather gy. 
siderably, being .40 and .84 for subtests 7 and 2, respect’ p^ 
Averaging these coefficients for the seven subtests ап [onh 
dicting the reliability coefficient for a test seven times ^» " 
(2, p. 283) gives an estimated reliability coefficient 0 ' 
compared with the obtained correlation of .94. е 3 
The correlations in the lower part of the table RR e 
necessity of stating the reliabilities of all measures W 
ers may use, since the various types of scores are pact 


equally reliable (7, pp. 122-3). 


jatiné 
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ТАВІЕ 8 
Reliability Coefficients 


Variables Equivalent forms 


Subtest 1 
2 
3 
4 58 
5 
6 


Total raw score .......... 94 
Standard score . 


Mental age : 91 
ГА кузем» ‚ 89 
RIQ MEET 93 


the Mos, is apparently based on raw scores for the age range 13-6 to 14-5, although 


anual does not make this clear. 
Probable errors of measurement are given for certain types 
of scores: (a) for standard scores Р.Е.м = 2.6, compared with 
2 reported in the manual; (b) for DIQ's: P.E. = 3.06; (с) 
for RIQ's: Р.Е, = 3.45. 


Factor Analysis 

In the manual of directions the authors state that they 
ave chosen the content in such a way as to “have a test more 
ighly saturated with a common factor or ability” (10, p. 1). 
П revising the TG test, for example, they eliminated those 
Subtests which appeared to measure a numerical ability, so 
_ Mat the present revision is thought to measure “general verbal 
Intelligence» (11, p. 1). While, of course, the number of sub- 
tests is probably too small and the reliabilities somewhat in- 


TABLE 9 
Intercorrelations (Form C in upper, Form D in lower part) 
Subtest 1 2 3 4 5 6 7 
1 
50 55 .50 45 36 
2 53 РА 67 0 — 81 56 
3 45 64 58 46 67 48 
4 S 50 48 41 56 49 
6 33 43 71 71 55 53 
7 49 .85 63 67 64 55 
16 46 55 40 48 49 
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adequate to give a satisfactory indication of the factor loadings 
obtainable from these tests, a factor analysis might give some 
indication of the extent of the general factor. Table 9 shows 
the intercorrelations, those for Form C being above and those 
for Form D below the diagonal. 

With few exceptions the intercorrelations for the two forms 
are of about the same magnitude. The first factor loadings 
and the communalities for each form were computed by the 
multiple-factor method (14). The obtained communalities 
varied somewhat from the first estimated communalities, whic 1 
were taken to be the highest r in each column. This 18, ° 


ТАВІЕ 10 
Е ecd Loadings and Communalities Jac Bach Farme €— 
Form C Е Form D — 
Subtest Ist App. 2nd App. Ist App. _ 2nd APP: — 
I 2 2 2 1 h? 
h I h I h ES 
1 a a G6 4 59 AM $6 4 
2 Wood € 7 жю 60 W 4 
3 7 59 в 58 ш 6 3 $ 
4 70 49 69 48 77 $9 76 в 
5 71 50 69 48 77 9 26 1 
р 84 71 .83 .69 .89 78 89 42 
68 46 166 44 159 35 46 oen 
E A 
course, to be expected with such a small battery of tests: m 


second approximation was made in each case. Only one E 
loading was computed since the correlations in the first i the 
matrix were all less than 4 times the probable error 0 
corresponding original correlations, making further ап 


It appears that little was gained by making the Owe? 
approximation since practically identical factor loading? aft 
obtained on both approximations. The factor loading? айй 
very similar for forms С and D. In general, subtests ^ д8 
7 are less saturated with the common factor than i5 t js? 
of the other subtests. This was verified by a cluster и 
(17), and also by the calculation of B-coefficients (D, del 

Since the subtests vary in their reliabilities a” 


THE TERMAN-MCNEMAR TESTS 57 


factor loadings, the question of the possibility of shortening 
the test without loss arose. Subtests 2, 3, and 6 have the 
highest reliabilities, and the highest first factor loading. Sub- 
tests 4 and 5 have identical factor loadings but the former is 
less reliable than the latter. Possibly a combination of sub- 
tests 2, 3, 5, and 6 would give satisfactory results. The inter- 
form correlation of scores on these four subtests was found to 
be .92, almost as high as the reliability coefficient of total raw 
Scores. The use of these four subtests would reduce testing 
time from 48 to 29 minutes, a saving of 40 per cent. 


Conclusion 


In general, the results of this analysis are very similar to the 
ata reported in the manual of directions, with the criticism 
: at the test may be too easy at the grade 9 level since it fails 
to discriminate between the mental ages of about 20 per cent 
А present sample. The suggestion is made that the test 
: be considerably reduced in content with little loss in 
reliability. 
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THE ROLE OF TESTS IN THE DIAGNOSIS AND 
CORRECTION OF SPELLING DEFICIEN- 
CIES OF COLLEGE STUDENTS 


FRANCES ORALIND TRIGGS 


American Nurses’ Association 
The Problem 


di AN EXAMINATION of the literature would seem to indicate 
at the college student who is a poor speller has received little 
ем а to do anything about improving his spelling 
soli s. This is in contrast to the encouragement given the 
inh student who is a poor reader through remedial classes 
dea ШЕ The complete explanation for this situation 8 not 
eee the following closely related observations 
Partially account for it: 
* Scientific study and diagnosis of spelling difficulties have 
lagged behind comparable work in reading; 
No clear-cut and easily applicable remedial tec 
spelling have been available; 
eachers of college students are 
oe were ever going to learn to spe 
7) oe 50 by the time he reached college. _ - 
ПЕ, to ca 15 growing evidence, however, that reading and spe 7 
and that У nothing of other language skills, are closely relate 
actually much can be done to remedy deficiencies in 


She at the college level. 
the ni at end, a remedial spelling pro 
e uy of Illinois during the aca 
Tequire techniques were sought for this program which would 
have t Students not only to read about spelling, but also to 
tele th © experience of applying the principles studied. It was 
Y using such techniques, there was some assurance 

59 


hniques 


convinced that, if a 
Il, he would have 


gram was set up at 
demic year 1942-43. 
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that the students could more easily apply the skills both in 
and out of their class work. 

For these reasons, a manual of exercises was the technique 
chosen. The spelling manual devised consists, first, of a dis- 
cussion of spelling in general, of the ways by which spelling 15 
learned, and of the types of skills involved in spelling; second, 
of a discussion of the principles of pronunciation with emphasis 
on those especially applicable in aiding spelling; third, of a 
discussion of word families; and fourth of a series of “spelling 
conventions” which help the students to see the system behin 
the spelling of many words. 

Answers to two main questions were sought from the 
dial spelling program: first, is it possible to improve SP 
skills of college students by use of spelling exercises W 
require the student not only to study the principles of go? 
spelling but also to apply them; and second, what kinds 0 
skills and abilities must students have who may be expect, 
to improve through this remedial technique, i.e., what рас 
ground is necessary on which to build spelling skills by U^ Б 
such a technique? 


reme- 
elling 
hich 


Procedure ; 
Announcements were made in Rhetoric I and II пошу 
students that they could apply for work in the remedial M 
ing classes. One hundred forty-nine students applied, of om . 
one hundred were accepted in the first remedial sections ОР. 
Approximately seventy students appeared at the first ale is 
of the classes. The work was carefully explained during od 
first session. Students were told that they would be eg 
to do the assigned work and do it regularly, if they were Ent 
to attend the sessions. It was expected that every $ Е ast 
would attend classes regularly once a week and spen ! on | 
two hours each week їп preparation of manual exer] 
Students were urged to come to the instructor's office for sr inl 
help previous to any class period if they had difficulty 
their assignments. chis 
Approximately twenty students did not return afte" -frys 
session, leaving about fifty in the four sections. Of thes? orf? 
twenty-two were called out with the Emergency Reserve 


| 
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Thus only about twenty-eight remained in the class long 
€nough to complete the work. 

Shortly before the first spelling sessions were over, a second 
Course was arranged to accommodate those students who had 
Not originally been accepted. This time some twenty students 
attended the first session, and about fourteen remained after 
the students understood the work which would be involved. 
Because of late applications, still another section was opened 
in which the students had to do twice as much work a week 
as had been planned originally. There were only about three 
Who were able to do this. Test-retest evidence did not indicate 
that these students were handicapped by having to work at 
Breater speed. 

Certain objective test data were available on these stu- 
ents. Scores were available on the American Council on 
ducation Psychological Examination. This is а. scholastic 

aptitude test having two types of scores, “L,” and “Q. The 

| “score purports to be indicative of language facility and re- 

to the student’s ability to do work SE dicar 

social such as course work in English, ааа ie 

ies oe The Q-score purports to be in i ч» 

Such ms facility to do work requiring quantitative IMRAN, 
Is required in science and mathematics. 


п addition to these Scores, scores on four other tests were 


Fs i . . 
of Pres an informal spelling test of the dictation type, one 
Clerical Test, and a 


© recognition type, the Minnesota Ule 4 : 
‘es test. The recognition spelling test giver was the spell- 
*<Ction from the С cooperative English Test, Form O, testing 


g 
abi i . B 
lity to recognize which of several spellings of a word is 


Cory, : 
286, he Minnesota Clerical Test has two sections, one 
on consists 


n 
unt and one on numbers. The numbers а И 

е i i iri the same. 

the „mns of numbers in pairs. If the pair 1$ exactly Р 


VE su | | c 
Similar Ject is to check it. The names section of the e E 
i i i i ires bo 
Spee his test is closely timed and thus requ 


tests and accuracy. The phonics test has two parts. Part I 

Ц te ше Student’s ability to divide words into syllables; к 
sts his nm Е 

е his ability to sound words according to a somewhat 


ы 
° 
check on extent of comparability of these two groups was made. 


Phon 
1 
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simplified arrangement of the usual dictionary key to pro- 
nunciation. : 

Scores from these tests and from certain clinical data indi- 
cated to some extent the types of difficulties which individual 
students had. There were those students who had good basic 
language abilities and skills as shown by a high score on the 
“L” of the American Council on Education Psychological 
Examination, names of the Minnesota Clerical, and the usage 
and vocabulary subtests on the English test but low scores 
on the spelling subtest and the phonics tests. The problem 10 
such a case seemed to be to make the student aware of the 
need for accurate spelling and to show him how to apply his 
skills by any of several techniques. 

The tests also revealed those students who had a ро 
facility in language, but who had never developed language 
skills. There were also those students who probably do not 
have the general ability and potentialities to develop ы 
language skills necessary to succeed in college. 

Students attended class for eight weeks for one hour б 
week. Each student was given in dittoed form spelling ents 
cises from the manual described earlier (Frances Oralind Trigg 
and Edwin Robbins, Improve Your Spelling, New York: 
and Rinehart, 1944). Individual conferences with studen; 
allowed the instructor to individualize somewhat the WO" | 
the manual to fit student needs as shown by the informal дең 
noses made from the type of work done both in and out of c 2 

Those students who cared to take retests were give” pe 
ferent forms of the same tests which they had taken ae p. 
beginning of the work. "These tests were then interprete геї 
them in individual conferences, There аге two types ° P n$ 
pretation which can be made from such retests: interpretat" y 
which apply to individuals only and interpretations арР ШУ 
to the group as a whole. Individual interpretations аге m "- 
of value in guiding the further growth of the student, а stt“ 
Tam = of Ms an intimate knowledge pen in 
tegis ah e E IBS tg зва udin, cM 
ow general trends which result from Г tio” 


. 1 c 
work. Both serve as a basis for evaluation and modif 
of procedures. 


tential 


a 
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Results of Remedial Work 


| The dictation spelling test was built to illustrate the prin- 

ciples discussed in the manual. The students at no time 
Studied the specific words in the test. Test-retest evidence 
Indicates a range from a gain over the remedial period of ten 
Words to a loss of one word, with a mean gain of 3.6 words, 
It is probable that results from this type of test most nearly 
reflect the ability of the student to do the spelling task usually 
Tequired of him. 

Test-retest evidence on the clerical test was interesting in 
that there was a greater gain on the names section than on 
the numbers section. The range of gain on the names part 
Was from 36 to 0, with a mean gain of 17 words. The range 
of gain on the numbers section was from 33 to minus 14, with 
3 mean gain of 13 items. This group of students originally 
had markedly higher scores on the numbers section of the test 
than they had on the names section. On the retests, this 
difference was not so evident. Gains on this test probably 
Indicate an improvement in ability to look within the word 
and recognize word parts rather than in ability to recognize 
the word only by its configuration. It is this type. of skill 
Which is used in proofreading and in reading where it is neces- 
VD to distinguish between words of like configuration such as 


Physiology” and “psychology,” “insulation” and “installa- 


tion » А А “ n" and 
«oD, and in many cases such simple words as “the 


than,” “also” and “solo,” and others. This type of skill prob- 
ably should not be over-emphasized because it might adversely 
аПесү reading skills. However, a balance between work of this 
ind and work on skills required in normal silent reading will 
Probably result in improvement in both reading and spelling. 
Gains were also evident on the phonics test. On the syl- 
abification section, Part I, the mean gain was ten words, with 
аапге of from 22 to minus two. On Part II of this test, the 
llity to sound words, the range of gain was from 27 to minus 
m With a mean gain of 11 words. A gain on this test, when 
dents Рапіей with gain on a spelling test, suggests that stu- 
alg; . "9t only have learned the tools of word recognition but 


аге beginning to apply them. When these same skills 


b 
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were measured by oral reading, it became even more evident 
that students not only had learned them but actually were 
putting them into practice. 


The Reaction of the Students to Remedial Work 


It was interesting to note the reasons students gaV 
registering in this course. In terms of the stated motives 0 
the students, they might be classified as follows: First; Фаг. 
were the students who were merely curious to know what t 
work would be like, but who did not care to put time J 
remedial work. Second, there were the sincere students y 
wanted to improve their spelling skills, but who actually d. 
not have the time available to work through the йал 
Many of these students were carrying heavy schedules = o 
actual work to help finance their education. This tYP^ or 
student is the one who is most severely handicapped by S 
verbal skills. Our university curriculum requires а gine pal 
of verbal work, yet it takes these students who have РОО! vo he 
skills longer to do the work; therefore, they do not Me end 
time to put on the remedial work, and the longer they J be 
on their class work, the less chance there is that they ше 
able to put in the extra time on improving their skills. a 
ти illustration, surely, of the old saying “them that has, & had 
ER NE а group of very sincere students oat clas 

Work, and who did excellent, cons!$ 00! 


ides Some of these students were handicapped end к. 
scholastic aptitude and did not gain as much in ig celle”, 
their efforts warranted; but most of this group made p 
чани as measured by both daily written work 1 
in their c i 

ourses and by standardized tests. € 


Att ee: Я o T 
he four weeks’ point in the remedial work, f 


the stud , : : ra 
skills 1 ents of the importance of consciously try!DE e the 


ea : : А a 

Stier, ad tk remedial work to class e n F. 
asked the student i "ne class t 2 

formal five- s to write during the ^o 


medial mich я expressing their бү” able 
Notice any im » indieating whether they had cim yeh 
provement i hat d Jat 
number of the re 
are given below. 


e for 


n their spelling up to f 


$ в : a 
actions, written both at this tim 
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_ The real purpose of this letter is to thank you for your help. 
The value of your spelling course showed itself clearly in my 
last theme for Verbal Expression. Though it was one of the 
longest compositions, it contained fewer errors than any pre- 
ceding paper. I misspelled only two or three words. It 
seemed almost unbelievable to turn page after page without 
an error. 

It will, as you said, be some time before I am able to realize 
the full benefit of your instructions. But already I can print 
legibly and at a reasonable speed. My spelling 1s improving, 
and one may see something in my way of doing things which 
Tesembles organization. My enunciation (thanks to your 
advice to visit the speech clinic) has shown some improve- 
Ment. lt will continue to develop since now I have the rudi- 
ments and need only practice. . 
$5 these things you've done for me against my own objec- 

Ons. [t would have been easy for you to let me go when 

Was determined to give up. It was some time before I could 
appreciate this work of yours. Now І can see what it has done 
and will do, so I want to apologize for my lack of character, 
and thank you for all you've done for me. 

jee 

A I have been a student of the experimental ‘cna m 
i for the past four weeks. In that time Ше has 
ot ja T transition of confidence within me ina phages 
Shan ndling and working with the English rg sue 
Sige may not be outwardly apparent a t E Pr pu 

» m but I’m sure time will bear out that the 

ё Improvement in this respect. 

Y one regret, in regard to this course, 
Weeks in length. 


i is that it is of only 

eight 

——— а 

in From remedial spelling I have received an improvemefit 
Spelling. T have never studied related words before or pai 


uch attention to the way the words were pronounced. 


thie s simple things have aided my spelling. Before I took 


a ques I never thought of the different ways of spelling 
¢s—hand, ear, etc. 


—— 
When I started to the University of Illinois I was very 


Weak in : % think I could have been 
Much spelling. In fact I don’t thin Idn't learn to spell. 


Worse. It seemed that I just cou 

ШЧ? find out what was the matter. | I was offered rs 
ing Се to take this extra spelling course to improve my spe k 
in th Was very much pleased with the chance, 4 Ters 
Weeks course. “I have just finished four weeks of the eight- 
5 course and I am beginning to see more closely some 
asic fundamentals of spelling which I had completely 
before. I can’t say after four sessions that I am an 
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outstanding speller, but I do believe that I will be a better 
speller after I have finished the course. 


I know that the four hours which I have spent in spell- 
ing class have helped me a great deal. I have been doing 
better work in my regular English class and the letters whic 
I have sent home have improved. 

I still am a very poor speller; however, I am able to find 
some of my faults. I believe before the spelling classes are 
over my spelling will improve a great deal more than during 
the first four weeks. 

I believe these spelling classes should be given next Yer 
so other students may also have a chance to improve thelr 
spelling. 


I believe that the help I am getting from our spelling 
class will not only help me to overcome spelling troubles, but 
it will be a great help in obtaining exactness with all my other 
work as well In fact I have already been helped by the 
principal parts which we have taken up, mainly forming ? 
picture of the word I am hunting for. Yesterday, for ex 
ample, I had to write a theme about myself while I was : 
the process of being sworn in as a Naval Cadet, and Jwa 
bothered with the spelling of a couple of words I chose to un 

y sight spelling came to my rescue, and I was able to ¢° à 
ecent piece of work on my theme. ‘This is only one instance 


: ed 
that I remember because it was so recent and much depen 
on it. 


,. TII admit that after the first few classes of remedial spell- 
ing, and after seeing the long and seemingly difficult assign” 
ments I was disgusted with myself for enrolling. I had au 
told myself that I was almost infallible in spelling, but ur 
mother was very disgusted about the lack of phonics 10 Her. 
grade-school system and insisted that I was a poor SPP jj- 
Spelling came easy for me and I imagined that remedial Fey 
ing in college would be one continual spelling match, and is 
are fun. However, I found that the accuracy the wor here 
quires is helping me in many ways. I’ve discovered that t?” T 
are many facts about spelling I had never thought about ge 
believe that this remedial work should be included in pt 

hetoric and English sections, because many freshmen, “nine 
graduated from high school, lack the fundamentals, trahe ghe 
and background to spell correctly, and the thoroughness ? 


ll t 


The faculty of the English Department was, at 2” y 


aware of what was being done in the remedial spellin£ 


work and assignments will aid in every course. 
Reactions of the Е aculty to Remedial Work ime? 
os 
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They referred students to it, and twice the work of the re- 
medial spelling classes was described at English staff meetings. 

he cooperation of the faculty with the instructor in remedial 
Spelling was excellent. There are many indications that the 
Instructors welcomed the special help given these students. 
Many of them felt that they had very little time to give in- 
dividualized help in spelling, but that if such could be done, 
the results would be worth while. Certain comments of the 
faculty on individual cases are given below. 


Thank you for your note about Mr. X. He has spoken 

to me of the excellent help you have given him with his 

andwriting and with his spelling. I am greatly pleased with 

is progress, and with his attitude toward you personally. 

I shall be referring students to you in the future, urging them 

to take advantage of the opportunity of following your sug- 
gestions. 


Your course in remedial spelling has been of considerable 
help to my student, Jack Doe. Originally, he was by no 
means a hopeless speller; but his spelling was bad enough to 
handicap him in his work. The carelessness and the word- 
ignorance which caused many of his errors have been checked, 

think, by the work he has done with you. On his themes, 
at least, he has shown an increasing awareness of the necessity 
9! correct spelling. Part of his improvement has come, no 
Oubt, from his general development in language skills as a 
Whole, through his work in Rhetoric, and from his own in- 
tellectual and social growth; but your work with his spelling 
as unquestionably given him valuable help with that par- 
Ucular aspect of his training. 
т. Doe has not, of course, 
а perfect speller. That is too m 
eveloped an interest in words themselves and has come to 
Tealize the importance of thinking while spelling. It is this 
New attitude, I think, which will have the most bearing on his 
Continued improvement in spelling. А | р 
f other students have gained. from their work in remedial 
Spelling as much as Mr. Doe has gained, I think the course 
Certainly should be continued for the benefit of future stu- 
ents, 


been suddenly transformed into 
uch to expect. But he has 


You asked what results your spelling class had on my 
Student, Mr, X, 
ү n begin with, his spelling was very bad, though largely, 

think, through carelessness. Almost at once his home 

emes showed great improvement as he became more con- 
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scious of his needs, and by the end of the semester, he rarely 
missed more than one or two—usually easy—words in each 
of them. ` й ; 
If that improvement lasts, and if your other students did 
as well, I should certainly want to see the class continued. 


Miss 


I am pleased with the progress made in spelling by | 
alu- 


Blank and Mr. Long. The spelling grades alone do not ev 
ate the counsel and assistance you have given these students. 
Your remedial spelling is a worth-while project and shou 
be continued. 


Conclusion 

dy 5 
а 16 
ali- 


The major generalizations to be drawn from this 5t" 
that poor spellers can improve their spelling skills by 
medial technique such as has been described. This gener 
zation can be made more specific by some further comments: Е 

There аге rather complete records for ninety of the stu ye = 
of this group. A study of these records indicates the impor 
tance of careful attention to the reasons for the spelling di g 
culties. For instance, twenty-six students had poor spell! 
skills mainly because of carelessness, lack of the ha 
reading what had been written, and, in gencral, an 
that spelling is unimportant. Sixty-four students, bog 
lacked at least some of the following skills: They cO" 
divide words into syllables, nor could they accent words E of 
rectly. They had very little knowledge of the constructions 
words—that is, they did not know what suffixes and PF rds 
were. They did not realize what base words or root MT 
were—and when reading orally they miscalled words 0 
configuration. Thus it became evident that they "x 
methods for attacking new words. They also had little kn? 
edge of spelling “conventions.” Many of this grouP m рай 
only poor spellers, but poor readers; and many of the? je? 
poor English skills as measured bs the objective p" Pos 
them at the beginning of the year and by subsequent 
work. o£ 
Nee these records, it is possible to ep. : 
of remedial s sellin x a aad of these students s into 
count. In this ae си е general ability hy uer hat 

gard, it might be said in gener? 


" 
a 
if 


~ 
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there is some indication of measured general ability and if 
remedial work follows a careful diagnosis of difficulties, the 
Prognosis of success in remedial work will be good, assuming 
the student applies himself assiduously. Вис for the student 
Who does not have measured general ability, successful results 
cannot be universally predicted. However, it is always pos- 
sible that a student's poor scores on the general ability test 
may be due to lack of development of language skills. If 
there is time available, an individual ability test can be used 
to determine to what extent the student is penalized by the 
form of the test given. If on the basis of an individual test 
Potential ability is evident and if plenty of time is available 
for remedial work, satisfactory results may be forthcoming. 
Probably the major error made in this remedial program 
Was that it was placed, for most students, on top of an already 
Over-full schedule. Requirements of the remedial program 
Were heavy, These students are already the ones who have 


be Spend the most time in the preparation d nm vals 
Cause of lack of verbal facility, which is a greatly e: 


too] 
th Н à 
TOughout the university curriculum. 


Recommendations 

ad йе basis of experience with this remedial кы" 
е ертык РН, first, that the students who E ge apa P 
ni & of DM and their records examined at t ips к 
ability b Пе school year; second, that the reason ы deed 
Stateq e determined in each individual case; thir , tha hs 
Pass Engli udents if they яе 
be gi nglish; and fourth, that a special place in the curriculum 
Stu ven for remedial training as may be required. If the 


dene... =. 
be tah 5 disability is great enough, his whole program pir 
an tened to allow time enough to do the remedial work, 

d again that, where 


Such on well. It has been found time an à 
“Nt not арргоасһ is taken, the student's improvement 1$ e] 
ча шу in spelling but in other language skills as well, 
this improvement is carried over into his course work. 

AE th ае urther observation should be made. The motivation 
Student is а major factor in the degree of success he will 


— 
€quirement be made of these st 
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have in any type of remedial work but should probably receive 
special consideration in the remedial spelling program. The 
extent to which it is important for an individual to follow 
spelling conventions will probably be a determining factor 10 
his motivation for remedial work in spelling. Though the 
clinician or instructor working with him may realize that prob- 
ably no strict line of demarcation exists between reading, 
spelling, and other language skills, it may be difficult to con" 
vince the student of this fact. If he is aware only of his 
spelling disability and has “gotten by" this long, it may be 
somewhat difficult to convince him that he cannot always “20 
by" with no handicap to himself. It is therefore recommence 
that the well-motivated students, as well as the students for 
whom prognosis of success in remedial spelling is good, be the 
ones to receive attention first, at least while remedial tech- 
niques are being evaluated. МЕТ 

There is always the question of how much responsibility 
the university can take in developing sub-college Englis 
speling, and reading skills. "This, of course, is a matter E 
policy to be set by the school in question. However, it! 
suggested that, if it is possible to demonstrate that spelling СЯ 
be taught at the college level, the public schools may be he P 
to realize that it can also be taught at the lower educatio? 
levels. They may then take over the responsibility at ab 


level and relieve the college of the necessity of WO" 
about it. 


" 


DISCRIMINATIVE VALUE AND PATTERNS OF 
THE WECHSLER-BELLEVUE SCALES IN 
THE EXAMINATION OF DELINQUENT 

NEGRO BOYS 


JOSEPH CHARLES FRANKLIN? 
Civilian Public Service # 115, Laboratory of Physiological Hygiene, 
University of Minnesota 
ын psychologist working with delinquents in an institu- 
C setting is obliged usually to maximize the validity and 
the 4 of his findings in individual case and group studies with 
east expenditure of time, energy, and resources. Conse- 
е» he is most likely to turn to the supply of available 
T and, applying criteria. growing out of his purposes and 
E "e test goodness’ in relation to prospective testees, 
eet. those test materials which are most easily admin- 
, Scored, and interpreted. 
Жш intellective measurement the use of tests on subjects 
р from the standardization populations from which the 
Bre: derive, in one or more significant variables, involves 
cern with the attainment of valid and meaningful 
Measurement. 
ie The Cheltenham School for Boys, Cheltenham, Maryland 
a State Institution for delinquent Negro boys. The back- 
8round of the boys committed is commonly one of social and/or 
“tsonal maladjustment. Their previous life conditions are 


m na Sie з 
arked by broken homes, inadequate familial organization and 
The incidence, 


п à 

variation, poor supervision, and neglect ~ ‘ica 

Care, э » of sub-standard shelter, poveltys luck af me as 
9 sen €ven malnutrition, is preponderant. These children 

—~—Seusly retarded educationally, approximately at die 


n for cooperation in the ad- 


The 
Writer į 
is indebted 
ti L. Grummo т 
o Donald ssistance in the completion 


Minig 

trati 

of the ation 9f the S j 
udy, cales and to Charles W. Piersol fo 
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third-grade level, at the chronological age of fourteen and a 
half; and truancy, suspension and expulsion from school, and 
consistent failure largely characterize their formal education. 
At the practical level of mental testing of such subjects, 
awareness of and consideration for the implications of the state 
of psychological knowledge in such areas of theoretical research 
as the following are of methodological and evaluative impor- 
tance: nature and nurture, race and nationality differences 
rural and urban effects, equality of normal opportunity for 
socially, educationally, and intellectually stimulating experience 
or the lack of it, the fixity or flexibility of mental capacities 
and the organization of mental abilities. -— 
It is beyond the scope of this study to discuss the a 
ships between the conflicting conclusions of research in Rp 
fundamental problems and the construction and use and à 
terpretation of obtained results in the mental measuremen i 
Negroes. Highly useful references are provided in bi 
ographies compiled by Bean (1) and the editors of the Jow” 
of Negro Education (14). exe Gh 
Nevertheless, keeping the relevance of the basic issues . 


: М ns aval 
mind serves two worth-while purposes. First, survey of ith 


comparisons of partialled-out components or aspects p ms 
tellective functions. It becomes obvious that in these i le 
tests easily administered, quickly scored, readily interpret 
and suitable to our subjects are not available. 
Preliminary use and appraisal of various group 4 
vidual mental tests were made. It was found that the pn 
ler-Bellevue A $ A Scales provided maximally useful in Tifich" 
tion regarding mental status and facilitated needed qU? Je? 
tion of test results with respect to the fundamental PI? 
already mentioned. jm 


iza 
Wechsler did not include Negroes in his standard? 


EXAMINATION OF DELINQUENT NEGRO BOYS 73 


and specifically urges caution in the use of his test with non- 
whites. Nevertheless, the apparent and distinct advantages 
of the Wechsler-Bellevue Scales in classification, insofar as 
classification depends upon mental status, warranted further 
use and study of the test. The bias of the standardization as 
related to this minority group is a serious incongruity but sub- 
Stantially no greater than that involved in the use of other tests 
the results of which did not favorably compare in usefulness 
and meaningfulness with the Wechsler-Bellevue. The proper 
€xtension of the use of the Wechsler-Bellevue to Negroes de- 
Pends upon such data as those which this report in part 
Provides, 
Purposes of the Investigation 


In order to assess objectively the suitability of the Wechsler- 
Bellevue Scales for the intellective testing of institutionalized 
€gro boys, this study was undertaken to answer the following 
Questions: How does the test sift and sort the population as to 
mental level? Do the sub-tests positively discriminate among 
the subjects as they are classified within the various mental 
evel categories? What are the patterns and trends of per- 
9rmance of the total and sub-groups on the sub-tests? Is 
the Suggested use of a short form warrantable with this 
Population? 
Procedure 


Two hundred and seventy-six boys were given the Wechs- 
ler-Bellevue (both Verbal and Performance Scales) during 
43-44. The average institutional population during this 
Period was about two hundred and seventy. For the most 
рай boys were routinely tested shortly after admittance but 
pome Were especially referred for testing for purposes of classi- 
fication from among those admitted prior to the initiation of 
© Program of intellective testing. 
he Wechsler-Bellevue Scales consist of eleven sub-tests, 
me of which is the Vocabulary alternate in the Verbal Scale, 
Which was not used. The five Verbal sub-tests depend heavily 
„Pon language for administration and for subject responses. 
“е primarily involve abstractual, conceptual, and general- 


74 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


izing mental functions. According to Wechsler as reported 
by Rosenzweig, Bundas, Lumbry and Davidson (10) they are 
described as follows: | 


1. Information: consists of questions formulated to tap 
the subject’s range of information on material that the average 
person with average opportunity should be able to obtain for 
himself. " 

2. Comprehension: measures the use of "common sense 
and judgment in situations described to the subject. Success 
on this test seemingly depends upon the possession of a certain 
amount of practical information and a general ability to use 
past experience. 

3. Arithmetical Reasoning: measures mental alertness as 
well as ability to handle practical calculations. . 

4. Memory Span for Digits: measures immediate memory 
for digits forward and backward. 

5. Similarities: measures ability to discriminate bet p 
essential and superficial likenesses; to generalize and thin 
abstract terms. 


ween 


та” 


The five Performance sub-tests require the subject t° such 


nipulate concrete materials and to perform certain tasks ame 
; : = i 5 

as arranging pictures and assembling object forms. The 

authors describe them as follows: 


6. Picture Arrangement: detects ability to comprehend " 
“size up" a total situation. . 

7. Picture Completion: measures ability to differen 
essential from unessential details. ‘oning 
. 8. Block Design: a test of general intellectual function 
involving both synthetic and analytic ability, but weg ns- 
considerably with ability to solve problems in spatial relatio" 

9. Digit Symbol: measures speed and accuracy of learn 
new associations. 

10. Object Assembly: measures insight into spatial rel 
ships of familiar objects. 


tiate 


ation” 


Each sub-test contains items which are related to odes 
ponent mental function and the items are arranged ™ , jpt? 
of increasing difficulty. Scores on sub-tests are convert? on of 
“weighted” scores which make possible direct compar" per 
the various sub-test performances. Separate Verbal a” да 
formance [.Q.’s are obtained by summating the appr me 
sub-tests, and these in turn are combined in an 07-2 
surement, the Full I.Q. 
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Results 


The chronological age range of the 276 subjects was from 
9.63 years to 20.13 years with a Mean of 14.6 and a S.D. of 1.56 
years. Results for the total group are given in Table 1. 


TABLE 1 
Average Performance for Entire Group (276 Cases) 
Mean Median Xm S.D. 
76.5 76.6 926 15.39 
76.2 75.8 .869 14.45 
804 82.9 1.094 18.19 


On the basis of individual test results the subjects were 
&rouped according to Wechsler: Normal (91-110); Dull Nor- 
mal (80-90); Borderline (66-79); Mentally Defective (below 

Test results for these groups are presented in Table 2. 
Omparisons of measures of central tendency and dispersion 
May be made since the age distributions within the sub-groups 
rey identical. These results are summarized in 
able 3, 


TABLE 2 

Performance Data of Sub-Groups According to Mental Level 
Mean Median S.D. 
IQ. LQ. Mean 
98.0 97.3 529 
95.3 954 7.12 
100.8 100.8 6.86 
85.1 84.7 3.37 
82.6 82.4 7.26 
90.1 89.5 6.05 
72.9 73.4 3.94 
73.1 729 747 
M 78.8 79.5 7.62 
Y 55.8 560 7.53 
стра 608 60.0 820 
crformance 60.7 60.3 10:62 


Discriminative Value of the Sub-Tests 
& Wechsler, Israel, and Balinsky (13) and Lewinski (5) have 
es positive discriminative values of the sub-tests of the 
Chsler-Bellevue Scales in differentiating between the various 
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TABLE 3 
Age Data for Mental Level Sub-Groups 
aS = — = ———— 
Gron N Ba Mean S.D. S.E. 
p co age Mean age Mean age. 
Normal ...... 52 1113-1863 148 1.63 218 
Dull Normal . E" 9.63-18.63 14.5 1.64 208 
Borderline ............. 90 11.13-18.63 144 1.34 141 
14.6 1.64 196 


Mentally Defective ..... 70 10.11-20.13 


intellective levels. Their studies, however, were done with 
quite different samples from that with which we are here 
concerned. 

In order to ascertain the discriminative values of the sub- 
tests in differentiating between subjects categorized ОП the 
basis of total test results, the differences in mean weighte 
scores, the standard errors of these differences, and the critica 
ratios were calculated. "Table 4 shows that all of the sub-test? 
discriminate between the various levels with the exception © 
three: (1). The Digit Span did not satisfactorily distingus. 
the Normal from the Dull Normal subjects, (2) the ре 
Symbol did not significantly discriminate the Dull Norm? 
from the Borderline, and (3) the Picture Arrangement | 
not significantly separate the Normal from the Dull Norm? 
While the results generally agree with those of Wechsler: oath 
and Balinsky and with those of Lewinski, they differ at sever 
points. The former found the Digit Span test of question? 
value in discriminating between Borderline and Defective $ ate 
jects whereas in this situation the same test does discrim! у 
significantly between these two groups. The latter one ов 
significant discrimination on the Digit Span between all gr^ te 
In this study, however, the Digit Span failed to different 
significantly between the Normal and Dull Normal group? 


Patterns of Sub-Test Performance v 


Inspection of the sub-test performances (see Table 5) "m 
that for the entire 276 subjects the five best-performo^ jap 
in the Performance Scale with the exception of Block eI 
the Similarities test of the Verbal Scale placing fourth in * f rb? 
of the first five. Accordingly, Block Design plus a 


a ——.... 


Ve 
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rbal sub-tests with the exception of Similarities ranked in 


the lower half of the ten sub-tests. In rank order the three 
highest, i.e., best-performed, sub-tests were Object Assembly, 


TABLE 4 


Discriminative Values of Sub-Test Performances Between Mental Levels 


N 


Sub-test groups Difference T И С.В. 
Information А 
Normal—Dull Normal ........... 2.50 Bi 16.6 
Dull Normal—Borderline 1.17 26 4.5 
Borderline—Mentally Defective ... 114 17 6.7 
Comprehension 
Normal—Dull Normal ........... 2.75 46 6.0 
Dull Normal—Borderline aie die кз» 1.05 28 3.8 
Borderline—Mentally Defective ... 2.21 23 9.6 
Arithmetic Reasoning € 
ormal—Dull Normal ..........- 2 55 45 5.7 
Dull Normal—Borderline €"— 1.33 42 a 
Borderline—Mentally Defective ... 1.91 37 Р 
Digit Span 
Normal—Dull Normal ........... 75 ES M 
Dull Normal—Borderline ......... 1.16 37 т 
Borderline—Mental Defective ..... 1.75 35 i 
Similarities 
ormal—Dull Normal ........... 1.73 * H 
Dull Normal—Borderline ......... 1.12 3 a 
Borderline—Mental Defective ..... 2.67 31 А 
Picture Completion ý 
Normal—Dull Normal ..........- 1.81 4 1 
ull Normal—Borderline ... 1.29 39 72 
А Borderline—Mentally Defective ... 2.82 3 2 
icture Arrangement 
ormal—Dull Normal ........... 1.15 e. i. 
ull Normal—Borderline ... " 144 35 95 
Borderline—Mentally Defective ... 3.33 35 
Object Assembl 
y 2 
ormal—Dull Normal ........... 1.36 p? m 
ull Normal—Borderline ......... 144 P 1 
" Borderline—Mentally Defective ... 3.25 я К 
lock Design " 
Normal—Dull Normal ........::: 2.35 2 se 
ull Normal—Borderline ........- 1.80 J a 
p Borderline —Mentally Defective ... 2.20 К д 
igit Symbol " 
Normal—Dull Normal ........--- E 52 i 
ull Normal—Borderline ........- 15 a EA 


orderline—Mentally Defective ... 


Picture Arrangement, and Picture Completion; the three low- 


est 


» Le, most poorly-performed, were Arithmetic, Information, 


and Block Design. Quite clearly, performance materials are 
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TABLE 5 
Data and Rankings in Performance on Sub-Tests of Mental-Level Groups 
pe сыы еы ЗР 
Ranking of Mean SE. 
1. Information 
Noc, opgoen eii 9 5 742 IAE 
Dull Normal » 10 64 492 1.84 їз 
Borderline _........... 10 90 375 112 12 
Mentally Defective ... 9 70 2.61 1,00 u 
"Total! 225 avait 95 276 441 2.38 . 
2. Comprehension 
ee à 8 m mM 3 
Dull Normal и 6 64 6.79 200 43 
Borderline ............ 6 90 5.74 1.21 19 
Mentally Defective ... 7 70 3.53 1.57 17 
Total: „нанос ана 6 27 6.15 288  * 
3. Arithmetic Reasonin 
ormal ы 8 52 7.73 2.15 2 
Dull Normal 9 64 5.18 259 7] 
Borderline 9 90 3.85 2.58 26 
Mentally Defectiv 10 7% 1.94 22 "9 
Воо сно 95 276 441 32 c 
4. Digit Span 3 
МОР авина надар 0 8 7.48 255 39 
Dull Normal 7 64 6.73 2.28 23 
Borderline. m 7 90 5.57 2.17 27 
Mentally Defective ... 4 70 3.82 228 — "6 
пота ise Uv 7 276 5.76 2.61 
5. Similarities 35 
Normal аена ллы 6 5 9.32 э m 
Dull Normal . 4 64 7.59 187 19 
Borderline 4 90 6.47 187 4 
Mentally Defective ... 5 70 3.80 198 37 
otal” cese asas 4 276 6.59 2.79 
6. Picture Completion 29 
Normal .............. 3 52 9.71 21 732 
Dull Normal 3 64 7.90 2.53 22 
Borderline ........... 3 90 6.61 2.11 32 
Mentally Defective ... 6 70 3.79 2.69 20 
TET MORI 3 2176 6.78 3.35 
7. Picture Arrangement 28 
Normal .............. 2 52 1040 20 5 
Dull Normal ......... 2 64 9.25 2.47 25 
Borderline ........... 2 90 7.81 2.33 25 
Mentally Defective 3 70 4.48 2.12 19 
ees 2 2% 779 3.16 
8. Object Assembly 5 
тори 1 5$ 105 20 я 
Dull Normal ........- 1 @ 97 2% 4 
Borderline ...........- 1 90 793 25 4 
Mentally Defective ... 1 70 4.68 2.93 21 
Tatil ve куста» 1 276 797 3.52 
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TABLE 5 (Continued) 


Ranking of Mean S.E. 
Sub-test group sub-test N weighted score D. mean 

9. Block Design 
Normal scies sexes 5 52 9.38 2.28 32 
Dull Normal ......... 5 64 7.03 2.19 27 
Borderline _........... 8 90 5.23 2.09 22 
Mentally Defective ... 8 70 3.03 1.76 21 
pl X PES 8 276 5.90 3.03 12 

10. Digit Symbol 
ОЕШ aux ossis ars ciate 1 52 8.25 1.97 27 
, Dull Normal ......... 8 64 6.63 1.66 21 
| Borderline ........... 5 90 6.55 144 15 
Mentally Defective ... 2 70 4.58 1.87 22 
Total) КНН 5 276 6.33 2.09 43 

— 


more efficiently handled and at a higher level than verbal 
materials. The results pertaining to the performance of the 
entire group on the sub-tests together with rank order of each 
of the ten sub-tests are set forth in Table 5. 

For the purposes of ascertaining the patterns of performance 
for each of the various mental-level groups the mean weighted 
Scores and their standard deviations on each of the sub-tests 
Were computed. The data are tabulated in Table 5 and pre- 
Sented graphically in Figure 1. The striking similarity of the 
Curves for all groups—regardless of intellective status—indi- 
Cates systematic and consistent variations for the population 
\1 organization of mental abilities and hence in their de- 
Velopment. 

For the population and for all 
8tound of general information an 


mental-level groups the back- 
d the mental alertness linked 
With 


th the ability to perform mental mathematical computations 


Constitute a special deficiency (Information and Arithmetic). 
© subjects were uniformly better able to comprehend or “size 
ish between essential and 


up? ЫБ ue 
ut total situations than to distingui ‹ 
essential details and parts of common objects and forms 


ee Arrangement and Picture Completion). Character- 
t ‘cally low performance on Block Design indicates poor syns 
“tic and analytic abilities in dealing with more complicated 
sroblems of spatial relationships as contrasted with ability to 
Ye problems of simple spatial relationships in assembling 


CUu 
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familiar objects for Object Assembly, which was the best-per- 
formed sub-test in all groups. | 
Some differences, however, are noted in the profiles in 
Figure 1. Information is consonantly low but exceeds Arith- 
metic at the Defective level, falls at about the same place at 
the Borderline level but falls below Arithmetic at the Dull and 
Normal levels. Digit Span exceeds Arithmetic at the low 
three levels but lies below Arithmetic at the Normal leve!- 
MEAN WEIGHTED SCORES 
отежа ero B, 
Information D| BJD N 
Comprehension 
Arithmetic 
Digit Span 


Similarities 


Picture Completion D Bo 
Picture Arrangement 
Object Assembly 
Block Design 

Digit Symbol 


Ficure 1 
Legend: - - - Average sub-test performance required to obtain Fu 
of 100 (14-6 yrs.) 
D Mentally Defective 
B Borderline 
DN Dull Normal 
N Normal 


и LQ 


rest 
The Similarities sub-test exceeds all other Verbal ao avel 
with the exception of Comprehension at the Norm.) ‘ect 
which is higher. Digit Symbol is about the same 4° “+ al 
Assembly at the Defective but falls far below the latter 
other levels. de 


"3 В on’ f 
Scatter—variability of performance achievement am n 


t 
- gtat™. 
sub-tests—in the Wechsler-Bellevue is associated with tic d^, 
maladjustment, neuroticism, and psychoses. Diagno® fail 
ical signs are related to patterns of sub-test success T. 


(3, 6, 9, 11, 12). Work in this area is in the expe" 
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Stage and the findings reported while not conclusive as to rela- 
tionships between psychometric test patterns and mental illness 
are suggestive. It is a matter of conjecture as to what extent 
psychopathology or psychological maladjustment influenced 
the range and level of sub-test performances of the subjects. 
It may be presumed, since few of the subjects examined could 
have been regarded as psychotic, that presence of clinical fac- 
tors does not seriously mitigate against interpretation of the 
data according to organization and level of the mental abilities. 
It is noteworthy, nevertheless, that examination of the test 
Profiles of groups above the Defective level discloses that in 
Sub-test performance five relate positively, two negatively, and 
two indecisively with Wechsler's (12) diagnostic pattern for 
adolescent psychopathic personality trends. 

The consistent and paralleling variation in sub-test per- 
formance of all subjects regardless of mental level raises im- 
Portant questions relevant to (1) the study of race differences 
її intellective abilities and (2) the relationships of systematic 
lower-level performance in tests of intelligence by minority 
groups to the extent to which success depends upon such factors 
as education, training, and experience (4, 7,14). It may be 
that the group patterns of sub-test performance reported here 
reflect relative handicaps in mental development rather than 
Manifest strengths and weaknesses of intellective functions. 
Fewer or other depressants to maximal mental development 
May exist in the white population on which the Wechsler- 

ellevue Test was standardized. Investigation is needed to 
discriminate the sub-tests in terms of the degree to which edu- 
Cational and social experiences and achievements are prerequi- 
Site to differential success in sub-test performance. 


Use of the Short Form of the Wechsler-Bellevue 


Rabin (8) has offered an abbreviated form of the Wechs- 
r-Belleuue Scales. Using the Comprehension, Arithmetic, 
and Similarities sub-tests and computing the total weighted 
Score by dividing the sum of the weighted scores of these three 
Sub-tests by three and then multiplying by ten, Rabin re- 
Ported correlations of .95 with the results from administration 
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of the ten sub-tests. It was his opinion that the regional and 


educational homogeneity of his subjects rendered his choice of 
sub-tests a good one for a short form of the Wechsler-Bellevue 
Scales. The author stated that because the Short Form 15 
primarily a verbal test it might not prove satisfactory for use 
with persons with a non-English language background. Rabin 
advised further use of the suggested Short Form with other 
groups of subjects for experimental purposes. 

In order to investigate the suitability of the use О 
Short Form with our subjects, the data were analyzed accor” 
ing to Rabin's method. All subjects were native-born with а 
common English linguistic background. 

According to this method I.Q.'s differed significantly нош 
those deriving from administration of the ten sub-tests for à 


f the 


TABLE 6 
Comparison of Results: Short Form and Full Wechsler-Bellevue Р, 
Mean 1.0. Mean 1.0. Mean SE. СЕ 

Group М ‘full test. Rabin’ О ЮВ 
Normal „оын 52 980 972 - 8 141 Es 
Dull Normal ....... 64 85.1 80.0 «5,1 1.18 43 
Borderline ......... 90 72.9 68.5 -44 1.02 41 
Mentally Defective . 70 55.8 514 -44 1.08 74 
TOP эуре кызыт» 276 76.5 72.3 -42 58 i 


mental-level groups with the exception of the Normal. Fo, 
the total of two hundred and seventy-six cases the Mean dy 
yielded by the short form was 72.3, which was significa” q 
lower by 42 LQ. points than the Mean 1.0. (765) ы 
from administration of the full test. In every mental ce? 
group the Short Form resulted іп a lower І.О. than oe 
sub-tests. In Table 6 data pertaining to the analysis 21° Epor 

It is concluded, therefore, that the use of the Rabin atis 
Form of the Wechsler-Bellevue Scales is not a steady ог ll” 
factory substitute for the ten sub-tests of the Wechsler che 
vue with the subjects examined. Caution dictates *" yos? 
Short Form should not be used with subjects resembling shot 
examined in this study. It appears obvious that f : pje 
Form should not be used in the mental examination 9 sitit 


whose verbal abilities are inferior to their performance ? 
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Summary 


The Wechsler-Bellevue Scales for individual mental testing 
were administered to 276 institutionalized delinquent Negro 
boys. The chronological age range was from 9.63 years to 
20.13 years, with a Mean of 14.6 and a S.D. of 1.56 years. 

The study was undertaken in order to report the results of 
the use of the Wechsler-Bellevue on this population, to investi- 
gate the discriminative values of the ten sub-tests of the Scales 
among the various mental levels, to summarize the trends and 
patterns of sub-test performances of the population and of the 
subjects grouped according to level of intellective ability, and 
to examine the suitability of a suggested Short Form of the 
Wechsler-Bellevue Scales for the mental measurement of in- 
Stitutionalized delinquent Negro boys. 

1. Results of the administration of the Wechsler-Bellevue 
Placed 19 per cent at the Normal level, 25 per cent at the Dull 
Normal, 33 per cent at the Borderline, and 23 per cent at the 
Defective level.? 

2. With the exception of the Defective group, the Per- 
formance I.Q.'s exceeded the Verbal I.Q.’s by 5.5 points for the 

ormal group, 7.5 points for the Dull Normal, and 5.7 for the 
Borderline group. Over-all, the Mean Performance І.О. ex- 
ceeded the Mean Verbal I.Q. by 4.2 points. Me 
. 3. The sub-tests of the IW. echsler-Bellevue Scales discrim- 
Mate significantly between the several intellective levels (as 
егіуе from the full test) with the following exceptions: Digit 
Pan did not prove satisfactory in distinguishing between the 
ormal and Dull Normal subjects, Digit Symbol between Dull 
Ormal and Borderline subjects, and Picture Arrangement be- 
tween the Normal and Dull Normal. 

4. There is marked similarity in the patterns of perform- 
ance from mental level to mental level. The group as a whole 
Shows striking disparity of achievement on the sub-tests. 
с €se differences in performance have i wer study 

i i 2 teristic = 
= etal differences. Those sub-tests characteristically per 


entages in the higher mental levels would 


2 A . . 
т А considerable increase of perc ^ 
oF Vt if greater weight were attached to Performance achievement at the expense 
1 4 
erbal in determination of the Full 1.Q’s. 
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formed at lower levels should be studied further in order to 
evaluate the role played by previous life conditions in their 
successful performance. 

5. Consideration in interpretation of the reported ipee 
should be given to the fact that non-whites were not include 
in the standardization of the Wechsler-Bellevue Scales. Some 
uncalculated error of measurement may have resulted from the 
presence in the subjects of states of negative adjustment, 0 
which there are indications according to the positive clinica 
signs developed by Wechsler and others. TT 

6. The Short Form of the Wechsler-Bellevue by which t 5 
Full I.Q. is derived from performance on three of the ten dl 
tests (Comprehension, Arithmetic, and Similarities) was P 
suited to mental measurement of the individuals examine" 
Evidence indicates that the Short Form should not be oa 
with individuals whose Verbal abilities are inferior to their ^ * 
formance abilities. 
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of 


тара” 1 eriods 
Scores of 60 adolescents living in foster homes and dependent for КУЛЛА 


of time, were correlated on the Revised Stanford-Binet, Form L, and the he IQ. 
Bellevue Scale. "The study confirmed the significant correlations between ^^ sler 
ratings on the two tests, but, unlike the findings of previous studies, the ! ы 
Bellevue I.Q. tended to be lower at all intelligence levels, especially so amont re 
dren with Wechsler-Bellevue 1.0). of 110 or higher. This confirmed AB i 
practical experience that the Wechsler-Belleuue Test appears to be poor, ay 


| an Chi 
Havighurst, Robert J. and Hilkevitch, Rhea R. “The Intelligence of India? social 
dren as Measured by a Performance Scale.” Journal of Abnormal 4" 
Psychology, XXXIX (1944), 419-433, ian trib 
, In order to find out the ways in which the children of several India! е, 20 
varied from tribe to tribe and from community to community within а PA rang, 
also to compare their scores with those of white children, 670 Indian childre hel 
ing in age from 6 through 15 were tested on a shortened form of the G7@¢ са 
Point Performance Scale. The Arthur Performance Scale was used because P Indie 
studies have shown it to be relatively culture-free. It was found ha nity д e 
children did about as well as white children, and that tribal апд comming одо 
ferences exist Just as in various groups in a white population. here M not ig 
indication that children from tribes little influenced by white culture а Ind? 
so well on the test, but there was no evidence to support the stateme 


nt that fadia 


Оше work more slowly than white children. It is concluded that m use 
d рел а performance test is а better instrument than a test requi 
е English language. Lorraine Bouthilet. 


MEASUREMENT ABSTRACTS 89 


Holzinger, Karl J. “A Simple Method of Factor Analysis.” Psychometrika, IX 

(1944), 257-262. 

A simple method for extracting correlated factors simultaneously is described. 
The method is based on the idea that the centroid pattern coefficients for the sec- 
tions of unit rank of the complete matrix may be interpreted as structure values 
for the entire matrix. Only the routine centroid average process is required. 
(Courtesy Psychometrika.) 


Klugman, Samuel F. “Test Scores for Clerical Aptitude and Interests Before and 

ae a Year of Schooling.” Journal of Genetic Psychology, LXV (1944), 

To determine whether test scores for clerical aptitude and interests, and the 
relationship between these two, remain the same after a year’s schooling, 207 white, 
female, native-born students in commercial courses of a vocational high school were 
tested and, after 2 semesters’ training, retested on appropriate portions of the 
Strong Interest Blank and the Minnesota Clerical Aptitude Test. A comparison of 
scores from the 30 oldest and a like number of the youngest indicated that the 
general improvement in scores noted for most subjects is probably due to schooling 
rather than maturation, since no reliable difference between means was found. 
Correlation between scores on the same tests one year apart revealed high relation- 
ship for clerical aptitude and substantial relationship for clerical interest. Vernon 
S. Tracht. 


Krugman, Morris. “Recent Developments in Clinical Psychology.” Journal of 
Consulting Psychology, VIII (1944), 342-352. | 
wo general trends in clinical psychology during the war period are observed 
by the author: 1) Halt in research on new clinical techniques, and 2) Great advance 
in experimentation in and use of short procedures including group tests and screen- 
ing methods. The Army's mental hygiene units are “child-guidance” clinics (for 
Soldiers), emphasizing test patterning and diagnosis, factor analysis in evaluation 
OF test batteries, and increased interest in projective techniques, especially the 
orschach and the Thematic Apperception Test. Abbreviated individual and group 
techniques are being developed for them. There is a corresponding loss of interest 
In personality questionnaire tests. Clinical psychologists are emphasizing diagnosis 
and neglecting psychotherapy. Ё. С. Bell. 
mdr 
Richardson i “The Interpretation of a Test Validity Coeficient in 
foe cde M sd Fa Selected Group of Personnel.” Psycho- 


Metrika, 245-248. Ё : 

he M Жш сә of a test used to select personnel is defined i terms 

of total effectiveness of the group thus selected, as compared with chance selection. 

e formula developed requires the use of an estimate of the ratio A arem 

effectiveness of men selected to the average effectiveness of men m sel erted by 

€ test. The predictive efficiency of the test varies directly with the таап ^ of 

this ratio and also directly with the percentage rejected. (Courtesy Psychometri a.) 
E 

Sadow i Є tical Analysis in Psychology of Education: Com- 

nly, Michael An Ма port and M nstrüctoré Driving Power." Psycho- 


metri, —256. : а 
бе ж iilos am derived for such concepts as stimulation of 
Student by instructor, student-instructor rapport, and driving power of instructor, 
In terms of the student's and the instructor's foci of attention, their strength of 
Concentration, and the intensity of the presentation and of the reception of details 


АЕ nds the assumption of normal distribution, the mathematical 
ter., Under р {оп yield conclusions on summary integral 


n be used to estimate the resultant effect of 
imple factors acting independently within the in- 
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i à m i ute 
dividuals forming the educational team. No claim is made as to ше гарм 
truthfulness and reliability of the psychological postulates used at the begi 
stage of the mathematical analysis. (Courtesy Psychometrika.) 


Spinelle, Leo and Nemzek, Claude L. “The Relationship of Personality Test чан 
to School Marks and Intelligence Test Scores.” Journal of Social Psychology, 
XX (1944), 289-294. iE Jes 
Results of a study undertaken to investigate the usefulness of the = Mist 

ventory of Interests and Activities for prediction of success in school showec er 

with junior-high-school girls, the measures yielded by Link's scale do not p! in 
direct value for educational guidance.” It appeared from the fairly high санаа 
tion between intelligence quotients and school marks that the intelligence quo an 

could be used for group, but not individual, prediction of scholastic succes. D 

that the Link Inventory should be considered as an objective questionnaire E tal 

information to serve as a basis for discussion in personal interviews in a me 
hygiene program. Lorraine Bouthilet. 
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Staff, Personnel Research Section, Classification and Replacement. Branch, 
General's Office. “The New Army Individual Test of General Mental 
Psychological Bulletin, XLI (1944), 532-538. З 
A new individual test of general learning ability was prepared if 1068 

many requests from psychologists in the military services, especially those, à 

in Special Training Units, Replacement Training Centers, and Army hospi 

convalescent centers. Seventeen verbal and non-verbal tests were tried АА 
reliability estimated according to the Kuder-Richardson formula, and маке 

carried out with the Army General Classification Test as the criterion. Т! 

tests and three non-verbal were chosen on the basis not only of gu le 

siderations but also of several practical requirements making the test арр andar 

Army use. The test was standardized, and norms are given in terms of sta 

Scores and Army grades. Lorraine Bouthilet. 
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Wallen, Richard. “Some Testing Needs їп Military Clinical Psychology. 
logical Bulletin, XLI (1944), 539-542. В 
Tests developed in civilian life are sometimes not applicable to mil 
especially in the task of testing recruits. Most published tests are tO 
dependent on a high level of reading ability, and too much time 15 
scoring and interpretation. A test for recruits should have easily und 
directions, the performance required should be simple, and the reli 
validity should be based on appropriate norms. It is possible to construc e 
test because the problem is primarily one of discrimination at only опе pilian 
the trait continuum—that is, of determining men who are not suitable ora : 
service. Since the purpose of the test is to weed out the grossly асур! a giver 
viduals, items to which a large proportion of the population respond IP Jorat” 
way are most useful. Promising results have been obtained in a few eX! 


studies. Lorraine Bouthilet. j 
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Wellman, Beth L. "Binet LQ. Changes of Orphanage Children: A Re-An? гё 

Journal of Genetic Psychology, LXV (1944), 239-263. ively; weh 
. A pre-school and a control group of 47 and 44 children, respect! iod whe 
given the Stanford-Binet tests at the beginning and end of the project per as 40 
ranged from 77 to 972 days. The mean age for the pre-school groun Q. ? 
months as compared to 40.0 months for the control group. The mean The road 
pre-school group was 86.9 while that of the control group was 83-5- иге? 
reaffirm the original study, indicating that the pre-school child with regu гов!" 
ance, and in residence more than a year, made significantly better p а si? 
intelligence than the child of equal initial intelligence, and in residence 10 
period, who did not attend pre-school. Betty Steele. 
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Wherry, Robert J. “Maximal Weighting of Qualitative Data. Psych ali 
IX" (1944), 263-266 rely jo 
, A method whereby biographical or other questionnaire data of а Po crt 
tative nature may be used to predict success or failure on an indepence 
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is presented. The method is not new but the present least-squares derivation and 
the transformation equation for punched card coding were not available in the 
literature. The proper weights are found to be proportional to the per cent of 
Passers in the various categories. The method is suggested as a suitable substitute 
for non-linear approaches in connection with purely quantitative data as well. The 
implications of reweighting in connection with multiple regression are discussed. 

he lavish use of degrees of freedom makes cross-validation extremely desirable. 


Ourtesy Psychometrika.) 


Wherry, Robert J. and Gaylord, Richard H. “Factor Pattern of Test Items and 
Tests as a Function of the Correlation Coefficient: Content, Difficulty, and 
Constant Error Factors.” Psychometrika, IX (1944), 237-244. 

A dilemma was created for factor analysts by Ferguson (Psychometrika, 1941, 

6, 323-329) when he demonstrated that test items or sub-tests of varying difficulty 

will yield a correlation matrix of rank greater than 1, even though the material 

from which the items or sub-tests are drawn is homogeneous, although homogeneity 

of such material had been defined operationally by factor analysts as having a 

Correlation matrix of rank 1. This dilemma has been resolved as a case of 

ambiguity, which lay in (1) failure to specify whether homogeneity was to apply 

to content, difficulty, or both, and (2) failure to state explicitly the kind of corre- 
lation to be used in obtaining the matrix. It is demonstrated that (1), if the 

Material is homogeneous in both respects, the type of coefficient is immaterial, but 
if content is homogeneous but difficulty is not, the homogeneity of the content 

can be demonstrated only by using the tetrachoric correlation coefficient in deriving 

the matrix; and that the use of the phi-coefficient (Pearsonian 7) will disclose only 
the non-homogeneity of the difficulty and lead to a series of constant error factors 
as contrasted with content factors. Since varying difficulty of items (and possibly 

Sub-tests) is desirable as well as practically unavoidable, it is recommended that all 

actor analysis problems be carried out with tetrachoric correlations, While no 

One would want to obtain the constant error factors by factor analysis (difficulty 

sung More easily obtained by counting passes), their importance for test con- 

truction is pointed out. (Courtesy Psychometrika.) 
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PHILOSOPHY AND PRACTICE OF PERSONNEL 
SELECTION 


HERBERT A. TOOPS 
Ohio State University? 


By definition selection implies more candidates than jobs, 
a choosing of the most fit. In boom years and wars selection 
wanes; it waxes in depressions and peace. Following the war 
it will become important again. 
_ It has become obvious that much of our material progress 
is due, on the one hand, to a very few expert people who are 
able to invent such things as B-29’s, radar, dehydration, and 
Penicillin; and equally, on the other hand, to a multitude of 
Joe’s, Bill’s and Sally’s whose skill of hand, keenness of eye 
and sureness of touch, in small things, just as surely is an 
expertness of its own. Some kinds of people do each of these 
respective kinds of work better than others. Subdivide and 
specialize industry as much as you will and still there will be 
more work for each of these kinds of people to do besides all 
the more supplying work for а third class of experts, the man- 
agers, the Henry Ford’s, the Henry J. Kaiser 5, the J. F. Lin- 
coln’s, and others of lesser publicity and prominence. Expert- 
ness is important in all these realms. — | 

Selection is both positive and negative. When looking for 
traits that are rare—in consequence of which we pay well for 
them—we wish to include as many as possible of the desired 
traits in one man; we look ideally for the one man of all men 
who most completely can fill the bill, the one man who includes 
in his make-up all the positive virtues. Of Tom, Dick, Joe, 
and Sally there are a myriad; hence they are paid chiefly for 
their time rather than for their pattern of abilities, and here 
We may seek only to exclude a certain few undesirable or nega- 


a . . H 
10n leave with the National Roster of Scientific and Specialized Personnel. 
95 
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tive traits such as dishonesty, unpunctuality, sickness, lethargy, 
and the like. We can afford—or so we often think—to hire 
them on an actuarial basis of “hire ten and fire two” since “not 
so much risk is involved in any individual mischoice" and “if 
they are not expert we can quickly and easily train them.” 
We also rely on the fact that what one workman fails to 
produce another may make up for by greater diligence; the 
shortcoming of one thus is offset by the superiority of another. 
In the creative, or inventive, and the managerial realms, ОП 
the other hand, the weakness of the one—whether superior ОГ 
subordinate—is less compensable, and in fact is more likely to 
result in a weakness of both. Yet even here the idea has 
potential merit: Could one pick “superiors” to work in раш 
or in the general case in teams—so that the strengths of the 
one overcome the weaknesses of the other, and vice versa the 
too conservative tendencies of the one curb the too radica 
tendencies of the other; the inventive tendencies of the 2 
stimulate the productivity of the other, till in the end, v 
the years, both come, like long-wedded couples are suppos”. 
to do, to resemble each other highly in all the good and virtuoU? 
traits? In fine, what persons should work with what person 
what roommates room with what roommates, what persons 
friends and pals with whom, and who should marry er 
We should consider such questions in terms not only of t? 


future. Its statistical difficulties are mainly a 
notation. We have, so far, a dearth of studies along ut 
lines. It involves questions not only of comparing pr? ol 
persons in their present or cross-section aspects, but also 
considering them in respect to their productivity ап 
probable future trend. 4 a 
About the course of growth curves we are not 50 certain g 
we once were. Thanks partly to war, we are not such ж 
in-the-wool hereditarians as formerly. In war—which оё! 
dentally has come to America with distressing regularity ond 
the decades of her existence, itself owed to war—the ser ave 
and third generations which fight the succeeding war 
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in each succeeding crisis little precedent to guide them. Hence, 
in these latter days, now that scientists are recognized more 
for what they are capable of doing than merely as so many 
additional units in the supply of cannon-fodder, more and 
more innovations are being tried out. We discover that illit- 
erates, not suffering from illiterate intellects, can be made 
“literate? in a matter of weeks; that most of the color-blind 
can be made color-seeing; that those deficient in the ability to 
see in the dark can be taught to develop “cat’s eyes”; and that 
plodding farmers’ sons may become great heroes of the air 
force. Do these not shake our faith in predestination pro- 
nouncements of a generation ago? 

We sometimes lose sight of the fact that man, with his 
superior cerebrum, is the most adaptive mechanism there is, 
extremely sensitive to the world about him, and particularly 
to that portion of his environment marked off as “the people 
with whom he works and lives.” So true is this that for every 
man who stumbles or falls, we, as practical psychologists, look 
for the woman in the case; for every divorce we suspect the 
Spouse; for every turnover we suspect the foreman; and for 
every corporal’s failure we suspect his sergeant. In a very 
real sense the traits of one’s fixed-relation associates thus are 
one’s traits. Enrolling such other-person’s-data in parallel 
columns of the data book after the “personal” data, one has 
the makings of a two-curves profile involving this social re- 
lationship; amenable to all the techniques to which any data 
may be put, and with some inherent niceties, such аз, for ex- 
ample, entering data by pairs (the paired associates’ respective 
Scores in a given trait) into a multiple-ratio regression equation. 
Compensation of traits would be revealed, perhaps, by opposite 
Signs of the two associates’ scores in a given trait. 

We are becoming more concerned, too, about what are traits. 
If they are not inherited, or not so much inherited, what 
then? Do they boil down to some physiological tendency such 
as the fund of usable energy which the individual possesses, 
Paralleled in turn by some simple index of amount of absorb- 
ability of vegetative tissues, or some set of more complex 
Teasons geared up with the glands or the like? 
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In addition, we have come, on the one hand, through m 
work of the clinicians, who concern themselves with € 
motivation and more recently with human reformability, ee 
on the other hand, through the work of the statisticians ү i 
have concerned themselves with those individual cases W e 
in correlation plots destroy the validity of the uet, 
question whether our erstwhile conception of personality ee 
added sum of its parts—or of its weighted parts—is the с d 
one. Everyone knows that some traits are more apri 
than others—for particular purposes, or at a ipe a nee 
especially at crises. This, then, is not to question e dus 
fulness to society—which as a whole always works ni re- 
actuarial basis—of the concept of the ordinary SE how 
gression equation. The statistician will never be ipee 
ever, with validities in the fifties and sixties and an е idea 
higher one. He knows that the difference. between h ‘js if 
and unattainable unity and his (always!) inferior d e 
part due to, or is associated with, individuals who do 
his simple hypothesis. | 

Add traits, either in numbers or of varied kinds, 9 єй 
and still the divergent individuals are almost as че a 
before; and the common observation is that despite all t же 
the multiple correlation coefficient is singularly idm 
Can this mean that our form of regression equation s As bio" 
implying in turn a wrong conception of the matter! cook © 
metrists we are little concerned with the inen at who 
drink and went all to pieces when his wife died, the 5 ania es 
suddenly turned truant, or the genius who sometimes А many 
from the two-room sod house on the prairie. [There 2 nsions-] 
more of such habitations (environments) than of p d 
In general, we leave those "situations" to poets. 


r both; 
t as 


ught О, 
е 

our conception to be enlarged so that these cases, аб pue 
those of normal school-children, of all ages, who hit =? е 
tomarily have been the subjects of our studies, are all н Jaini”? 
under the same formula? There is little merit in a , abe 
them away by naming them cases of shattered, pec we 
normal, or emerging personality. erat” 

Are there not indeed crucial traits which unless ОР 
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at least to a minimum degree render all others fruitless? 15 
it conceivable, for example, that any American without a cer- 
tain mathematical background (which few Americans possess 
but which many conceivably might acquire) could read Ein- 
stein no matter how intelligent, persevering, or able as a reader 
he is? This essential lacking, the product is zero, or virtually 
zero, no matter how favorable the scores in the remainder of 
the “causal variables." 

In the statistical treatment both the units of measurement 
—in their mathematical aspects—and the form of the "regres- 
sion equation" will surely be involved. Little is written on the 
matter now, but much more deserves to be and will be written 
about it in the years to come, particularly if a few mathe- 
matically capable or promising and ambitious people can be 
recruited to the psychological profession. 

Finally, to round off our background of the matter, is it 
not clear that the cross-section aspect of a trait which we 
Bet as the result of a test or inventory (qualification form, 
questionnaire or interview, for example) is only evidence, 
rather than fact, only a straw in the wind of how the individual 
&rowth curve blows—or grows? 

There are available but few growth studies of individuals 


in the several functions of physical growth, mental growth, and 


89041 growth, "hog ГО АШ een 
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of life such growths, predicated perhaps on a common, T 
ey “energy basic to the several growings resident mre 
ures а а 
imply that the child is father to the тап іп more t 


Year, у ; Id see 
t t is usef - at the child if we would $ 
useful then to look at the That old dogs 


than that old 
to keep track 
owth study 


Prefer fn for what he is and yet may be. 
dogs c 9 ‘earn no new tricks is more of a truism 
9E hi an learn no new tricks.) And it is useful 
S the i, Toughout life. Consequently a general gr e 
aay п “Ividual may be more revealing than any number а 
ое... У of cross-section variables which ordinarily we p 
Made or selective purposes. Quasi-growth studies Өк, 
Зиед Ык а consideration of the ages of the individual, c i 
Or the dates of certain happenings, h as of his severa 
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Present prognostic tests may be more so by name than by 
realization, particularly in the absence of long-range follow-up 


studies to convict us of our errors of “misappellation.” The 
ШЕ : 3 

value of an *evidence" is measured not by its name but rather 

ment 


by its validity coefficient. Thus we see that the improve 
of selection lies largely in the development of new techniques: 
Only a very few of its roots go back to the past. Our present 
practice accordingly may be found to be faulty, or be Биоуе 
up unduly only by actuarial considerations which, partly °F 
alone, save the day. 

The last point is worth belaboring. 

Two hundred men are let out of a modern plant, A, Jet us 
say, which is closing down on war production, on a Saturday: 
They have been trade-tested and it is known that 80% of 
them are “good workmen.” The other 20% of “not so 800 
workmen are able to drive rivets, bolt-up nuts, and in genera 
do any work for which they were hired, but with inferior spe? 
and accuracy. In peacetime they would be «non-hirable" b& 


s : We : art- 

cause too inefficient. Plant B is just starting up a new depa" 
up at 

"clot 


dis 


charged 200 are standing in line outside B’s employ! its 
door; also that in recent days the government has dou ed ! > 
initial order with В, so that the word goes down to the employ’ 
ment office to “Hire them quickly so that we can get them? 

work, We can fire those who don’t fit in. The governs 
will pay the bill.” So our obliging employment clerk € p 
the men in line, notes that there are 200, mentally c? cule 
since they want only 100, that а орак а half dollar wi ual 
the job nicely and moreover will give every man g nd 
chance at a job.” He makes a little speech to that effec те? 
hires the men accordingly. And all are happy, eve? € chat 
who get no job—since they had their chance—as though тег 
were a good in itself!—save the father whose infant " , pay 
badly needs at once an operation which only a job ae nis 


for. . . . . mn ý 
а ya is statistics and probability, not human ut pall 
social security.) So in fifteen minutes the impar" 


m 
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dollar responds to *Heads we hire this man; tails we don't" 
in its expected manner and soon one-half of them, 100 in num- 
ber, are at work. Silly? Perhaps! But what is the worst 
job that could possibly be done by our half-dollar under the 
circumstances? The answer evidently is: To hire all 40 of 
the “not so good” men. (Twenty per cent of 200 men equals 
forty men.) That would yield a selection efficiency of 60 per 
cent. The half-dollar would do the job as badly as that “only 
once in a million times." What is the best it could do? 
Obviously, to hire 100 of the 160 “good men,” and reject all 
40 of the “not so good” as well as 60 of the “good,” with a 
resulting index of efficiency of 100 per cent. This too is a 
very improbable occurrence. What is the most probable 
result? Evidently to hire 80 good men and 20 not so good 
ones, resulting in 80 per cent of efficiency of placement. (The 
same figure which the trade-tests revealed.) If classification 
experts, placement clerks, manning tables, job families, morale 
officers, and all the rest, can beat this figure, the excess—above 
the 80 per cent—can be credited to their efforts; and ifa lesser 
figure is obtained, their value then is all on the negative side 
of the ledger! : 

Let us carry this matter a little further back in time. We 
note, then, that much selection has occurred before the 200 
men started for the doors of our employment clerk early this 
morning. This may conveniently be said to consist of two 
types: self-selection and pre-selection. 

The self-selection consists in such facts as that only men 
able to walk showed up at the office; only men able to read or 
Possessed of relatives and friends who could read; that no 
Coffee-tasters, goldbeaters, astronomers, mind readers, college 
Professors or tight-rope walkers applied—indeed only people 
Who wanted work, this work which they judge, rightly or 
Wrongly, is like what they have done before or which they hope 
they may be able to qualify for. | 

The pre-selection consists in the effect which the ad, mainly 
Perhaps, had on the potential applicants. If the ad says “gen- 
tiles only,” few Jews will have the temerity to apply. If it 
Says “white or colored,” some colored who otherwise would 
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be hesitant may decide to apply. If it says “men only,” what 
woman will apply? If it says “$10,000 and expenses” what 
$200-a-month clerk will apply? Indeed selection is benignly 
affected, all unkownst, by a myriad of such considerations 
which cost little or nothing of effort or thought—and which, 
in passing, may have just as little to do with the efficiency ОП 
the job of those hired, the lucky ones we would call them 2 
mere four years ago, and perhaps still might do so with almost 
as much justice. 
_ That the patient gets well without doctor is counted à 
miracle and that he gets well in spite of the doctor is neve! 
counted a blessing! If the doctor is called, all the world 1" 
cluding the patient is satisfied that a great good has besa 
achieved! But we—those of us appointed to note an E 
better such things—cannot be satisfied merely because ош 
clients are contented! We find it necessary, useful, and right 
to look into the selection process with a critical eye: Ont 
fruitful way of doing this is to look into the implied statistics 
behind each of the current or possible modes of selection. 
might be well, however, to delay that consideration for 2 little 
in order to ponder another matter or two and particularly p. 
strictly statistical principle which grows out of the successi" 
hurdles method of selection? This method implies that ur 

the passers of a given examination are allowed to take the в 
sequent qualifying examination; and evidently this test PE. 
also be a statistical one, a sieving process applied to the p 
of the several individuals on a common trait-profile recor ; it 

their record cards; only those who qualify on the first. ЇЇ? 
being allowed to be considered for passing the next test; dle 
Dexetrai, the passing point of which comprises the next ш ne 
In high-jumping contests we do not necessarily rank every е 
exactly in the order of their jumping ability by ruling 007 $ at 
who do not (we did not say “cannot”) jump the bam act’ 
three feet, a difficulty measure we say, forgetful of other ete 
x m ias motivation of severe competition gu ot 
a record is broken. In “tests” of select! "jor 


the difficulty, but rather the validity, of the test is the ’ А 


f 
2T « : 1J” 
H (932) 16-2187 А. “The Successive Hurdles Method." The Person” 
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tant consideration. We get more or fewer jumpers merely by 
altering the height of the bar, by lowering or raising the passing 
point of the test. The all important consideration in selection 
1s what kind of persons got over the bar—passed the test— 
and are allowed to take, or are subjected to, the next test. 
Accordingly if a trait of low validity is used for the first sieving 
any successive sieving of humans in selection, however fine, 
cannot undo the effect of having let most of the better indi- 
viduals go into the discard. Still more concretely, the ends of 
all selection are at least two in number: 

1. To assure a quality of those selected which shall be 
materially above that of those rejected. 

2. To reduce a larger number of “applicants” to a smaller 
number of those recommended or hired. 

The former objective, of course, is the important one. It 
follows then that in attaining the second end, that of reducing 
the number of candidates, it is important to keep the merit of 
the retainees as high as possible and to maintain it there as long 
35 possible in the process of reducing the number of applicants. 
Briefly, it finally amounts to this: that the ideal selection 15 
achieved when the successive hurdles or tests are applied in 
descending order of their validity coefficients and that the ap- 
Plication of additional tests is stopped arbitrarily whenever the 
number of applicants decreases beyond a given minimum.® 

Ё the tests have no validity at all, as is true of our half-dollar 
above, then the order of application of the hurdles is im- 
Material. All that the tests accomplish in this latter case is: 


1. A considerable reduction in numbers by reason of the 
Application of the hurdles.‘ 
рас eem 


th NIE regardless of the number of persons who pass all the hurdles, it is decided 
lat all the hurdles (or tests each with their several passing points) shall be applied, 
willy nilly, the order of application of the tests makes no difference; the same in- 
viduals will respond to the total process in every permutation of order of applica- 
ton of the selective variables. We assume that the passers of two tests will be more 
а le than the passers of one only; but this expectation follows, the first test having 
°Sitive validity, only if the second test has a validity higher than zero; and it can 
тшу destroy some of the validity obtained if the second is of negative validity. 
The validity of two can be less than the validity of the best one alone also when 
t Weighting methods of selection we “overweight” the poorer test beyond its multiple 
theression, -weight. It is by no means axiomatic that “two tests are always better 
one. 

ti Jive such successive hurdles, each eliminating 50 per cent of the previous 
"tainees will reduce а field of 3200 applicants to 100. Two more, or seven in all, 


Will reduce it to 25. 
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2. A building up, in the minds of the applicants, of a belief 
in the fairness of the tests. [A feeling which unfortunately 
can be, and very probably will be, almost as great for low va 
lidity, or zero validity (chance) tests, as for highly valid ones.] 

| So long as they were believed in, for example, tests of mem- 
orization of Confucius were “good tests” in China, but any 
statistician could tell you that the character of the civil serv- 
ants of China under such a system could be no better than that 
of the average person able to master this first hurdle to political . 
preferment. A good rote memory obviously is a valuable asset 
of a statistical coder but surely is not all important for а high 
civil оћсег—ог at least most civil officers—of the state. This 
settles at one stroke any contention that a uniform patter® o 
selective traits—at least of those presently available—c9" 
be equally valid for all purposes, for all occupations for m 
stance. Traits are important—or so we now consider—™ 
terms of their weights in a multiple regression equation. І e 
our conception of the regression equation and some new ae 
Fin the relative importance of the traits results. At pres 
the traits employed by sociologists, census-takers, and news 
BApers £0 typify the social-composition of the individuals 9 f 
social body may have zero or even negative validit 
"EIE the extent of their correlation with an ind¢l 
сптецоц of ability in the occupation or undertaking in y of 
tion—and yet may serve just as well as апу more V2"! ° pet 4 
traits to quickly reduce the number of applicants to a pur. 
feasible for “more detailed examination (usually an inter"! 


and consideration before appointment." ion fof 
B . i i9 

| M econ selection of equals is at stake and compet iver 

jobs prevails, clearly justice requires that each perso” , |, joe 


S s + ape 
an equal chance of being selected. Of course this princip val, 


not apply to unequals, for the better man, other thing t 
should always be chosen. And if things are not eque Ke 
almost equal, then there surely is some social gain i * 
nouncement can be voluntary. Not every teache an г 
ample—who incidentally happens to have a husband у, а 
any Justice in giving up her job to an inferior, equally goat 
even slightly better teacher who happens to be bot us 


-— 
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less and jobless. No obvious improvement—and possibly 
Some loss—in the condition of the taught is foreseen as the 
result of such replacements. We are only in process of estab- 
lishing the morality that one's right to a job and to advance- 
ment is independent of his social and economic status.* 

The severity of the critical scores employed in the succes- 
Sive tests has a bearing not only on the amount and the quick- 
ness of elimination but also upon the validity of the sortings. 
To eliminate no one by a highly valid test is to eliminate en- 
tirely its validity as a selective agent, to cancel it out as if it 
had never been given. We are unable to state this effect in 
other than the very general statement? that the more valid tests 
Should have the more rigorous critical scores if there is any 
Variation of standard in the several hurdles. 

There was an intimation above that under some circum- 
Stances there was merit in an actuarial hiring of all who apply 
and in letting the test of the job decide who should be retained. 

П а cost-plus economy this is always feasible particularly if: 
« 1 The percentage of applicants who are above the minimal 

acceptable point” of competence is high. (In our example 
above, if only 50 per cent of our applicants instead of 80 per 
Sent were “competent” the impartial half-dollar could give us 
NO satisfactory selectees at all. With the number of compe- 
tents less than 50 per cent, the probability of such an occurrence 
's higher and higher as the percentage index descends.) 

2. There is some highly efficient method of quickly weeding 
Out the incompetents on the basis of their performance subse- 
Went to selection. 

These considerations yield us our first potential or actual 
Selection method. 

The Test of The Job 


If we measure day by day the rate of learning (or “prog- 


Te: » Ф - 

AE P uf beginners, say, by means of adequate work records, 
8 The war has given us a new morality about the matter of one firm or one 
Кесе gutting the market for good men, merely because it happens to be first, or 
ave the most money, and the like. 3 

Ri n important initial attack on this problem has recently been made by 
of Pardson, M. W. “The Interpretation of a Test Validity. Coefficient in Terms 
(1o4Kreased Efficiency of a Selected Group of Personnel.” Psychometrika, IX 

14), 245-248. 


ч 
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both a prognostic test and a trade test may be unnecessary. 
Relative indices of progress on the job may then be all the 
test we need. This is true because there is some warrant for 
believing that differential capacities yield individual progress 
curves, with distinctly different rates of growth, which differ- 
entiate at an early date. The rank orders of the growth curves 
of the apprentices at an early date thus approximate the final 
orders of competence. Our technical problem then 1s t? 
shorten the tryout period as much as possible. 

Let us assume that all of the new workers in a € 
tory are put through a sequence of jobs, first on the drill pres? 
then on the shaper, then on the planer, etc. It is clear, the» 
that the number of hours that it takes candidate A to complete 
a standard drill press assignment as compared with the hour? 
required by candidate B is some indication of which © m 1 
two 18 presently more competent thereon; perhaps some sligh 
indication, also, of which of the two gives more promise 
both were equally “untaught” at the beginning—of future Ыы 
fulness in that particular drill press department; that is to says 
possesses more “drill press aptitude.” At the end of the " 
suing shaper operations, which it is assumed follow in all саз 
upon the drill press operations, the cumulative number 0 a; 
required for completing both assignments gives а better ү 
sure of the “all-around” mechanical capability of the candi y 


ertain fac 


| J 
than the previously mentioned shorter “test” dealing э 
a single (somewhat more specialized) ability. део 7 che 


the greater the number of pertinent experiences include’ j in 
testing the more is revealed the “all-around” mechanic? rn^ 
genuity, or general mechanical ability or adaptability oF now" 
ing-power, of the (apprentice) learners. The questio" ріс? 
becomes a statistical опе: “What is the earliest period f W 0 
the cumulated competence scores can be made to СОГ” el 
at least a minimum limit, say .90, with ultimate comp?” есеб" 
The end sought is to minimize the time, work and money th? 
sary to make a valid decision of whom tö keep (becaus? ansl 
probably, of promotion) and whom to discharge or pt 
(because of inadequate aptitude for the work). TE 

of presentation of the work experiences, drill pres 


| 
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planer must have been fixed upon, of course, and be uniform 
for a particular entering group of apprentices, but may be 
altered on subsequent groups to place the most valid shops 
(those correlating highest with the sum of them all) into the 
earliest positions in the try-out experiences, as is necessitated 
by the successive hurdles method. On our initial experiment 
all must complete all assignments to arrive at an over-all cri- 
terion score for each. Employing the L-technique one now 
may easily ascertain how highly the first shop correlates with 
the sum of the success scores involving them all; how high 
the score of success in the first two shops combined correlates 
with the total success; how high the combination of the first 
three, and so on. The correlation will become high at an early 
Stage, particularly after the shops once have been arranged in 
an order of decreasing correlation of the several shops with the 
total success on all the shops. 

An alternative, and preferable, method is to place in first 
Or accepted position in our upbuilding composite job that task 
Which correlates best with the sum of them all; then, in second 
Place, that one of the remaining which raises the correlation 
most; then that of the remaining tasks which raises the cor- 
relation next most and so on. The eventual order is "that 
Order which maximizes the prediction of the criterion with a 
minimum of tests.” We have here a choice of utilizing either 
the multiple-ratio method’ or the L-technique, the latter being 
Preferable wherever simplicity is important. The former does 
not require the computation of all the inter-correlation coefh- 
Clents, which is an advantage if the number of elements to be 
Combined is large. If all the intercorrelations are available 
then the Wherry-Doolittle technique is appropriate and has 
merit for the purpose of selecting a minimum of “shops” to cor- 
Telate maximally with their total. This simultaneously mini- 
izes the time, and optimalizes the order of presentation, of the 
accepted shop experiences as a test of machine-shop aptitude. 
= 

7 Toops, Herbert A. “The L-Technique.” Psychometrika, VI (1941), 249-266. 


See also, Toops, Herbert A. “A Self-checking Technique for Shortening a Test.” 


* C. А. Bull., No. 92, Jan. 30, 1934, 2027-2040. : : 
3 Stead, Shartle, et al. Occupational Counseling Techniques. New York: 


American Book Co., 1940, pp. 245-250. 
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In all such methods some regression of the validity coefficients 
of the selected scale, on subsequent tryouts, is to be expected. 
The point at which the thus obtained correlation, by what- 
ever method, first reaches .90 or .95 say, will mark off both 
what shops it may be necessary to give to all candidates and 
also the desirable order of their presentation. The individual В 
score оп the accepted composite will reveal substantially 
whether a given individual does or does not possess sufficient 
*aptitude" to be allowed to continue. Where there are neces- 
sary sequences in learning skills—of which there are few 10 
either school or industry—the statistically dictated optim? 
order of presentation cannot of course be employed. 

| Let us suppose that аз а result of an adequate experiment 
in which a number of beginners have carefully been watche 
through to ultimate competence, it has been decided that thé 


us 
number of hours needed to complete the first five ш 
ing : à Р stl- 

reordered) operations is a highly valid measure for progno 
then: 


cating ultimate and “all-around” success. It is clear, 
that if we build five successive sets of norms of cumul 
progress as of the ends of these five several operations 
“normated” individual performances obtained from the Е 
itself on the five successive occasions of completion of an ? he 
tional project are progressively more and more indicative of í f 
ultimate worth or lack of merit of the individual apprente a 
concern. For solution of our statistical problem no HUN m 
other than the test of the job itself, is necessary. 
dustrial agencies (as of wartime) demand a decision 
be made quickly with only fair accuracy it may be mace qe 
the first operation, with somewhat better validity em of 
second and so on, the validity improving with the lengi the 
6 tryout.’ Ву the steps outlined we have discover” gnd 
minimal number of operations (involving their identit cost 
sequence) which, with the least possible wastage of supe gat 
time and overhead cost, enables us to arrange the ca” | rech 
— t 
"me. ы ыша, wee it is said that most souls are saved in пй add "e 
] . Tests or hurdles beyond the first highly valic: 


to the reliabili 
bility of the scale—and so to the “justness” of the exc 


sidered from t з t 
validity of я of the excludee or potential excludee—but 


ative 
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substantially in the order of their relative excellence—centile 
ranks, perhaps—for retention (and promotion). The above 
applies when even the poorest man turns out a salable product. 

But the problem is not quite so simple as the above state- 
ments would imply. “А man is worth the selling price of what 
he adds to the product less a reasonable profit” is an axiom in 
economics perhaps but vocational psychology is not quite so 
naive as that. Observation shows that the worth of a man is 
measured adequately neither by the wage paid him nor by the 
quantity of work he turns out. Both are only symptoms of his 
worth or value. The slow-learning but ultimately fairly ca- 
pable employee may be more of a “success,” from the employer's 
viewpoint, than the quick-learning person who is the delight 
of the teacher or foreman (the typical factory teacher). This 
statement follows if the employer values highly traits other 
than productivity, matters such as punctuality, dependability, 
or gentility. No series of measures can ever completely de- 
Scribe the man. The individual is what he is in the measures 
employed; he is something else when more or other kinds of 
measures or tests or aspects of worth are applied. The em- 
ployer then may indulge his preferences (or prejudices) be- 
tween such “types” as a “slow but accurate” individual and a 
"rapid but somewhat inaccurate individual.” Concretely, let 
Us assume that the first of two persons rates 40 for speed and 
60 for accuracy. If the arithmetical units are comparable, so 
that they may legitimately be averaged, the first individual 
then averages out to 50. Let us suppose that a second indi- 
vidual rates 60 in speed and 40 in accuracy. His average also 
1550. But the two individuals, thus seeming alike, each having 
an average of 50, are quite different in the employer’s eyes. 
The first will not get so much work done, but he has the merit 
that he will not destroy so much raw material; the quality of 
work produced will be better; and he probably will not require 
So much supervision and correction of his work. Such con- 
siderations are *values" to most employers. Inquiry on the 
Part of the U. S. Civil Service Commission, for instance, has 
Tevealed that business men are quite content with a lower 
typing speed in a secretary than that set as the required gradua- 
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tion speed by a majority of stenographic schools provided only 
that they can acquire people who do not make so many mis 
takes (even if they don't get so much done) which need to be 
corrected. In line with this, employers rate highly the ability 
to spell (to correct their boss’ mistakes). 

The above homely example indicates that success is not 2 
“unitary” concept; it cannot be measured in one dimension but 
can be only more or less adequately portrayed by a profile, а 
series of scores in a number of variables, measuring ideally dif- 
ferent “dimensions” of the applicant. Without tests, can W^ 
still measure the individual for selection in terms of a profile 
to the end that the profiles of different candidates may f 
compared and a choice be made? By making the permissib e 
assumption that in some occupations at least a man's past 
portends his future—which is a safe assumption in fields suc 
as teaching, management, and the professions generally, jury 
ever occupational growth requires considerable time and | 
continuous—this is readily done by a technique previous y 
published by the writer,” and may be briefly summarized 4 
to its principles or elements: 


The Test of Accumulated Evidences 


1. An extensive qualifications blank and specific we 
mendations blanks received from former employers or фас 
provide a great deal of concrete evidence, from various рт. 
as to an individual's success to date in his occupation а? 
adjudged promise therein. 

2. By the aid of official evaluators of such evidenc® we 
function as a standing jury in that capacity year after Y 
the lines of evidence are telescoped into quantitative sor 
some objectivity and validity, recorded as some por the 
(present practice) aspects of ability deemed important in 
job in question: intelligence, research ability, scholars eer 
the specialty, general scholarship, social intelligence, manag and 


and executive ability, special skills useful in the occupatio" 
the like. 


or 


1 


10 Toops, Herbert А. “The Selection of Grad Assistante? The person 
Journal, VI (1928), 457-472. е Selection о! raduate А5515 2 
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3. The resulting scores, when plotted, become the candi- 
date’s profile just as truly as if one had test scores, social com- 
position items, standard scores on a standardized progress 
schedule, and the like." The subsequent evaluation may now 
be by any of the general methods given below. In the article 
referred to, the subsequent evaluation was by the summation- 
of-traits-scores method, which may or may not be best for 
one's purpose, as noted hereinafter. 

Given, then, a profile of the individual, assuming all neces- 
Sary corresponding statistical implications, there are a good 
baker's half-dozen different ways of selecting talent, that is, of 
reducing the number, while increasing (as an expectation) the 
competence, of the retainees. All of these in common assume 
that the pertinent traits (or at least the traits on which he 
selection is to be made) have been observed, preferably objec- 
tively measured, on each of the candidates and that the scores 
necessitated correspondingly are available. 


1. The summation-of-traits-scores method 


1.1 Where the scores are added at gross-score weights of 1. 
This is the method customarily used in most school 
examinations where the sum total of the person’s merit 
is simply the arithmetical sum of all of the scores on 
the several sub-parts (sub-tests and items). This 
implies an addability of scores which seldom obtains. 

This method, contrary to popular belief, does not weight the 

sub-parts equally, but instead weights each item, and each test, 
proportional to its standard deviation. In many cases this 
may not be such a bad assumption, however, since the test 
which can produce the greatest spread of scores in general is 
the most valid. 

1.2 When the several Scores of a given person on several 
traits or sub-traits of the total examination in succes- 
sion are multiplied by a fixed series of arbitrary indi- 
vidual test gross-score weights. This is the “weight- 


a ue 
ing" method which is popul ith civi i 
g popular with civil service gen- 


її Тоорѕ, Herb Q9 а m : А 
surement, iV стл зоте Criterion.” Educational and Psychological Mea- 
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erally. It is likely that civil service frequently has 
greatly erred in the weight which it felt it was attach- 
ing to the scores on the several sub-parts in question. 

It is a safe guess that, the standard deviations abetting, 
many a man has been selected by civil service mainly 
on account of his handwriting or some other compara- 
tively unimportant trait. Such “errors” can be 
avoided by changing the original scores to ranks, 
standard scores, T-scores, or other relative measures 
in which the standard deviation of the different traits 
becomes a constant. Standard scores have a standard 
deviation of 1 while rank scores have a standard dev" 


ation of oma ; T-scores a о of 10, and so on, irre- 


spective of the variable measured. f 
The weights may be ascertained by the bids procedure © 
the ensuing footnote™ which standardizes the process of ascrib- 
ing the weights and replaces the presumably inferior judgment 
of a single person by the possibly superior judgment of а group 
of “experts.” . 
The weights may be obtained mathematically, а criterion 
being available, and thus have the superiority ascribable = 
least squares technique. It will be recalled, however, Фат 
a unitary criterion score may come from vastly different pro 
files of “success variables.” 
The most popular of selective methods fails then to dly 
well even criteria whose similar mode of compiling undoubte h 
weights the scales at least slightly in the direction of hi£^ 


v 35 : ep" 
rather than low, validity. It is perhaps the simplest conceP 
———— = po 

12 Tf B, be the importance actually assigned to standard scores, then т? ait 
where W, is the proper gross score weight to be employed in order that ersel 
shall be weighted with a true relative weight or importance of Вг. Conv ht 0" 
a gross score weight, IW; be arbitrarily assigned to Test 1, the true relative wert igna 
importance thus assigned is not W;, but B;— Ис. Clearly, В, and В: аге Bron the 0° 
то W, and We, or conversely, only when 6;=0:; or in general, only e mont 
of the tests (of sub-parts) are equal. Standard scores, ranks, and T-scor? 
others, have this property. ноп dB Я 

Clearly also опе may vary the length or difficulty of the examinatio! 0101129, «d 
to achieve approximately this end, a point of finesse worth further exp tanda 
If a test is lengthened to л times its present length its new and enlarge 


deviation is onzGVan-n (л — Г) ғ, where Fy is its reliability coefficient. cit. 
Toops, Herbert A. “The Selection of Graduate Assistants.” 02- 
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tion of the organization of traits, merely the sum of the 
weighted scores, and is suspect for that reason alone—as is the 
I.Q.; for what else, pray, could one do in the latter situation but 
take a ratio! Man prefers simple laws; Nature, perhaps! 


2. The successive hurdles method 
This, referred to above as a principle, is the method which 
is popular with civil service in times when the supply of candi- 
dates greatly outruns the possible demand for placements. An 
examination is given in common to the entire list of candidates 
who have qualified on certain preliminary minimum specifica- 
tions (health, freedom from arrest, etc.). A critical score is 
established above which all of those who *pass" the first exami- 
nation are certified as entitled to take a second examination, 
while those below are denied further opportunity to qualify. 
Successive examinations eventually whittle down a large list 
of candidates to a few who are certified for appointment. The 
author™ has shown that, when using this method, it is highly 
important that the tests be administered in the order of the 
most valid test first, the next most valid second, and so on 
down to the least valid last. The least valid and last test may 
then be practically a chance test. (It should not have a nega- 
tive validity coefficient, of course, for that would reduce, instead 
of improve, the mean talent of the retainees.) Accordingly, if 
chance is to have a hand in the selecting of candidates who 
eventually are offered positions, clearly chance should operate 
only on a group of people who as a result of preceding screenings 
have a very high degree of capability. This condition easily 
is procurable by the sequence recommended. Clearly it were 
better that all the tests should be equally valid (and ideally 
intercorrelate zero), but since this is impossible the principle 
stated is obviously the correct one. From the viewpoint of 
the function performed it is clear that an individual always is 
thrown gut by E single test, which in the nature of all rests has 
at least а degree of ЙН and a Validity which may er 
May not be very pertinent, ie, Valid, to the particular job 
i 


14 or ; by the Successive 
Hurdle! oops, Herbert A. “Sifting Civil Service Applicants, by 
SN Method,” The ни Journal, 1 (1932), 216-219. 
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which a given candidate is to fill.” This method had its origin 
during the depression when thousands applied for positions for 
which scores or at least only hundreds had applied before. It 
is the product of desperation! It is costly; and at best is not 
the method which one of free choice, cost being no considera- 
tion, would choose. 

For many a candidate his destiny is decided by only one 
or two traits (the earliest tested) of his entire profile. The 
average competence of 100 candidates chosen by such methods 
—if the selective traits are valid—of course is high? when the 
tests are administered as above indicated, namely in a decreas- 
ing order of validity, and with the more valid tests producing 
the greater (and earlier) elimination of the applicants. This 15 
the method employed by the Westinghouse Science Talent 
Search. Functionally, this method has much in common with 
the precise profile method below (q.v.). It has the merit that 
by setting a more severe standard the testing can be terminate 
briefly, thus minimizing its cost. 


3. The precise profile method 

In this method a certain number of traits are assumed to 
be highly (and equally) important to the extent that if an 
individual does not have precisely the skill-pattern, ог profiles 


established as important (possibly by preferences of the pro" 
spective employer) he shall not be considered. The WX 
Holle 


a large number of individuals having been punched into 
rith cards or Findex Cards, say, the subsequent sorting pets 
cess yields all those persons who exactly fit the “selection PIO” 
file.” If too many candidates result, another trait, of раи 
validity perhaps, may be added to the pattern to reduce € 
further the number of cases; or the subsequent selection may 
be subjective, based on “a detailed examination of the applic 
cant’s entire dossier.” (This implies a positive validity © 
15 Obviously the larger the roster and the fewer the referrals the кемер уе 


othe of the actual referrals, if the given selective method has 
16 The normal emi x " -—9 greatly 
its reliabili al expectation in lengthening a test is that this will incre’: prov 
ш е an ity and increase slightly its validity. One of the "easy" ways to її з 

е validity of а test is to shorten it by eliminating the least valuable item? chen 


of 100 or more items often will be found to have items of negative validity 
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efficient of the examiners’ judgments.) Only those who qual- 
ify on all the traits are acceptable. The machines are blind; 
they know no “humanitarian considerations.” The method, 
as outlined, presupposes that all the traits are applied without 
fail to the candidates; hence the order of sorting of the traits 
(if one was employed) can have no influence on the competence 
of the selectees. The method presupposes that the needed 
numbers of candidates have been measured on all traits, and 
the task is that of picking the “most likely” candidate for 
placement. 

If Findex cards, and quantitative variables, or qualitative 
Ones possessing intrinsic quantitative characteristics are em- 
ployed the chief fault of the method may be somewhat miti- 
gated by slotting’ on the “or more” basis in order to pick 
out, automatically, candidates who have at least the minimum 
Standard, or more, on each trait. Thus, any person who is 9 
in a trait is also slotted 8, 7, 6, 5, 4, 3, 2, 1, and 0; while an 
Person who is 10 is also slotted 9, 8, 7, 6, 5, 4, 3,2, 1, and 0. 
If, then, one sets, say, 7 as a minimum in a selective trait, all 
Persons who are 8 or above and 7 or above also will respond 
to the selector rods of the mechanism, as well as those persons 
Whose score is precisely 7. In the above case, if one sets 10 
as his minimally acceptable score, only the second candidate 
will respond to the rodding; but if he sets 9, or 8, or 7, say, 

oth candidates’ cards will respond because both are equal to 
9T are greater than these several limits, respectively, 
enough candidates show up, one m 
to 6, say, and find additional cas 
question to “fill one’s quota,” 


wants to pass some definite per 
this one trait. 


If not 
ay then reduce his minimum 


es who are 6 on the test in 
on the assumption that one 
cent of all the applicants on 
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many candidates show up, one may raise his minimum require- 
ment in the trait or traits in which it is known or is judged that 
such raising is likely to be most profitable (namely the trait 
or traits of greatest validity.) 

Some of the traits may be expressed as unvarying stand- 
ards; that is to say the particular category in the trait in ques- 
tion is the only one acceptable. Thus, one may insist on hav- 
ing a male teacher, females being totally unacceptable; or ап 
unmarried teacher; or a Protestant teacher; or any compound 
characterization of these, e.g., а married-man, or a married- 
man-who-is-also-a-Protestant. With respect to traits which 
are strictly quantitative, such as intelligence, scholastic aver- 
age, height, weight, etc., usually anyone above a certain mini- 
mum is deemed acceptable. Sometimes, however, an upper 
limit is established as well. That is to say, policemen must e 


at least 6 feet, 0 inches tall and not more than 6 feet, 4 inches- 
(This would imply the existence of a curvilinear relationship 
ure 


of the traits upon the independent measure (criterion meas 
of job success, for which selection is being carried on.) 
Hollerith equipment is employed similar conditions may а 
made to prevail either by multiple-punching one-column pel 
in similar fashion (this precluding tabulation) or, better 
the selection of particular packs of cards for successive sorting 
The multiple-sorting head device is essentially the exact PT? 
method. 

This “or more”-variation of the method resemble 
popular method used by housewives in picking maids, by Pe 
captains and army recruiting sergeants in picking recruits» 
employers in hiring workmen, and the like. It is freque”, iy 
used by placement bureaus to place individuals, the саг ue 
removed from the “live file” just as soon as a given p 
placed. It is the method employed by the National Ree at 
of Scientific and Specialized Personnel, with the variation 
an already employed person, if an “essential specialist, 
be referred to a prospective employer whose priorities we 


gs the 


rson 


rran” 


18 Only by an ade ; š 
quate follow-up, when all are hired (as in wart! ali 
шен Somn type can we ascertain the relative merit of such qualitative 4 ]ed£t 
ius It is not impossible that some of these are on the negative side 0 * 
umanity undoubtedly pays through the nose for its hiring prejudices- 
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Clearly the method presupposes a great supply of persons 
for a limited number of placements. Its essential difference 
from the successive hurdles method if the or-more punching or 
selection is employed, is a distinction without a difference. 
The fact that all the possible candidates have had all the “tests” 
(e.g., have answered all the questions of all application blanks 
and supplied all required credentials) and that the entire pat- 
tern is employed makes not a whit of difference. А too-low 
score on only one trait eliminates him sooner or later, but just 
as surely. If the result of the compound dragnet (applying 
all the hurdles at once instead of in succession) is unsatisfac- 
tory, the test profile may be changed until a “likely” candidate 
comes to hand. 

Its chief merit is its potential flexibility. The housewife, 
so inclined, for example, may interview all who answer her ad 
and if many respond raise her pattern of requirements to “get 
more for her money.” 

When the traits are set by employers ignorant of trait im- 
plications the result obviously can become little more than a 
means of securing a random selection among the available 
candidates and of indulging the whims and prejudices of the 
requisitioner. The best to be hoped for is that any such pat- 
tern in general has “some pertinence” to the jobs in question. 

Its great fault is that the candidate who is generally very 
superior but lower than the minimum in only one trait—who 
is 100 centile, say, in all traits but one in which he is, say only 
one centile below the established minimum—will fail to be 
selected. And since the selection is blind—i.e., the operator 
sees only the cards of those who respond to the selection, and 
is blissfully unaware of the “all but” cases—the conspicuous 
failures of the method—its shortcomings are never noted.” 
The Findex is limited to selection where only such numbers of 
cards (individuals) are involved as can be manipulated for 
hand sorting. Larger numbers could be handled by having 

19 This presumably might not be the case with Ke 
the “almost” cards would have a single solid card a 


holes if the card in question chances to be in the mids 
to be performed” cards. 


ysort or Speedsort cards since 
Cross a continuous channel of 
t of the “all but one operation 
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the cards filed in pre-sorted compound-breakdowns, which to 
that extent may defeat its own end. 

This might be called the fastidious method of selection. It 
reminds us of the diner who to quench his thirst must have 
wine; and not only wine but Chablis; and not only Chablis but 
Chablis of a particular vintage, 1927, say; and who moreover 
is willing to pay any price, including that of not drinking, in 
order to appease his whim! Unless the available supply of 
applicants is large the precise pattern may locate no one. This, 
however, oftentimes is the consideration which makes the 
method appeal to the politician who would nullify civil service; 
Since with a not too large roster an employer given that option 
very easily may set up a pattern of traits that none but the 
intended recipient of the proffered position will be able to meet. 

In practical application the practice is to establish by judg- 
ment an ideal pattern; and then to investigate the available 
supply of applicants by the Findex or by the more recently 
perfected Multiple Pattern Sorting device of the Hollerith 
machine, an attachment for the sorter which will sort out all 
cards of any given pattern not exceeding ten contiguous 
columns of the eighty-column Hollerith card. Then if the 
initial sort yields no applicant, or not enough applicants; the 
pattern may be altered to a less ambitious one, whereupon more 
applicants will come to hand. In theory the “less important 
qualifications are dropped, but who, in the absence of apP!" 
priate regression equations can say which are the "less im- 
portant” qualifications! 

Where prospective employers set the patterns, if they СО 
be induced to establish an order of decreasing desirability ps 
sorting the cards, and if the number of desired referrals coU 
be always specified also, or else be predeterminable by form" Е 
from the number of workers requisitioned, then the sorte 
would have a freedom which permits of lowering or raisin£ t a 
standard of competence in order best to fill the requisito" 
Even here "stereotyping" of orders may reduce greatly а 
chances of referral of a man who is substantially good but у 
in exact agreement with the stereotype. The short man wh 
on other scores would be a superlative policeman is out 9 и 
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so long as more than enough 6-footers respond to that universal 
requirement! And this despite the fact that modern auto- 
matics are supposed, for some purposes, to make all statures 
even! 


4. The minimum divergence from the desired profile method 

Having an “ideal profile" in mind the candidate’s profile 
of test scores can be matched against this profile and the candi- 
date then can be hired by reason of an index of his agreement 
or lack of agreement with the ideal profile in question, say “that 
of the job.” Probably one of the very best measures of the 
extent of such agreement is Sagebeer’s index. This index has 
a value 0 when the profile of the individual exactly matches the 
profile of the individual for which the selector is looking, while 
larger values of the index indicate greater discrepancies be- 
tween the candidate’s profile and the ideal. The method ob- 
viously assumes that the traits are measured scores and are 
comparable. Possibly the scores should be standard scores, 
thus making the assumption that equal standard scores in two 
Ог more traits are equal. The method will be most meaningful, 
perhaps, in large groups of applicants, all tested on quantita- 
tively scored tests which are highly reliable and as nearly 
unique as possible. Possibly it applies better where aptitude 
rather than achievement is the basis of the selection. The 
ideal of occupational selection, as of any other type for that 
matter, is that all applicants, not merely those who happen to 
be labeled by the particular magic name of an occupation, for 
example, should be scanned for selection. ‘The above index un- 
doubtedly would be too laborious for this purpose without 
Some machine method of solving the formula. This method, 
employing the index, has possibly never been formally used or 
used very much because of its newness. The method presup- 
Bio mnn 


?? Sagebeer's index is: а 
T= У(Х)? + GG -X9*t.. +s 
Where 
Xi, is the ideal profile score in trait 1, that established 
authoritatively by job analysis. 
X; is the candidate’s score in trait 1. 
The correlation coefficient between the scores of the candidate’s profile and the 
‘deal Profile would be an alternative index for the purpose at hand. 
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poses that every person has been compared with the ideal 
profile and a measure of his disparity therefrom has been ob- 
tained. Assuming acceptable comparable wnits for the several 
tests, this is possibly the most ideal of all methods, for general 
selective purposes. It tends to prevent an individual who 
is very lopsided from being chosen and yet allows deviations 
from the range of ideal scores to be considered—although at а 
(slight but appropriate) disadvantage. 


5. The predominant or outstanding merit method 


The essence of this method consists of the following opera- 

tions: 

5.1 On a sheet of cross-section paper, in Column 1 are 
entered the names of the several candidates. 

5.2 The headings of the several columns are labeled with 
the various traits on which the candidates are mea" 
sured. 

5.3 Into the compartment is written the most signifi 
statement (preferably quantitative) of the candid 
standing in the trait in question. This is done for 2. 
traits and all persons in turn. This then is the PI 
mary data table, or the Hollerith tabulator reproduc 
tion thereof. ; 

5.4 Considering now trait 1 only, опе may encircle 1” 
column 2 those statements which represent the most 
meritorious degrees of the trait desired in the candidate 
[In general these will be the largest scores; in y am 
cases, such as errors or time, they will be the smalle? 

scores; and in case of curvilinear relationship they ' 

be the scores that indicate greatest fitness for the T 
that is to say, the scores closest to the apex ° А 
parabola or curvilinear relationship line betwee? ] 
trait in question (X) and probable job success А 
Possibly the ten per cent ог so of highest Pes Ne 


cant 
3 
ate 5 


so encircled. The number so to be encircled V! 
pend both on the number of candidates and ОЛ 
number of traits. 


В alts 
5.5 The same now is done for the remainder of the ns 


А i 
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5.6 One may now either count and record the number of 
circles for each person (that is, the number of circles 
per row) or, as an alternative, may count up, for each 
person, the sum of the weights of the traits appropriate 
to the columns in which a given person possesses 
circles.?' Р 

The end achieved by this method is to give unusual weight 
in selection to those persons who possess unusual excellence її 
more than one trait. In other words, it is rather the antithesis 
of the method in which we set up minimal test scores and are 
Satisfied with any one who, so far as the single trait is concerned, 
is at least above that point. By this method we assure our- 
selves that the appointee surely will have several strengths— 
if any candidates possessing such are available—although he 
also may have some very fundamental weaknesses; that is to 
Say, he may be lopsided, but nevertheless he will be at least a 
‘near-genius” in some respects. The method presupposes a 
Breat scarcity of the kind of talent (genius) desired and im- 
Plies a desire to locate the “best” there is available. 

This method implicitly, rather than formally, is assumed in 
Case one goes out “combing the world" for unusual talent of 
any kind. Unusual talent is so scarce and withal comes so 
dearly that if one can get a near-genius in several traits, one 
Benerally is quite willing to overlook a number of minor or 
even a few pretty serious faults or defects. The adaptability 
Possible in a specialized industry often makes it particularly 
feasible to place an individual where his defects, from the view- 
Point of the job at which he has to work, will be practically 


no handicap at all. 
One may vary this technique 


of defects which one wants on a 
Consider for appointment him who has the fewest defects. 


his, presumably, formally approximates the procedure ordi- 
Пацу used in choosing diplomats and other public relations 
SS 


by encircling instead all scores 
ll accounts to avoid and then 


21 Тһе weigl determined by the bids-procedure, presumably. If 
Ше В% н сна j sodes blade, with the data table tacked in alignment to a 
(Wing board, the B's of the blade opposite the encirclements only are added. 
of th, sts is a convenient device for bearing the marginal f's into the body 

е table, 
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appointees where freedom from public offense is or may be a 
greater virtue than positive capability. 
Still another variation would be to encircle the strengths 
and then to box-in all weaknesses, the score for a person being 
the number of his circles minus the number of his rectangles.” 
Such a method would require a larger number of registrants 
than either of the last two methods to produce the same num- 
ber of selectees. It has the advantage that those selected have | 
high basic strengths and at most only a very few fundamental 
weaknesses. n 
Other methods are no doubt possible. The above, how- 
ever, comprise those which readily came to mind. 


Comparison of Methods 


A careful inspection of the above systems will reveal that 
there is no one method which at all times and in all places 
is superior. Each performs somewhat different functions an 
has somewhat different emphases. To ask which of the 
methods is better, accordingly, is analogous to asking the ques- 
tion, "Which is the better container for apples: boxes, bags, 9T 
baskets?" The only reasonable answer is “It depends on 
times, places, and circumstances.” 

In the practical situation the matter of motivation of | 
the potential employee is a portion of the total value t5 M 
be placed upon a method, as well as the excellence of the siev- \ 
ing which it produces. It may or may not be wise, for © 
ample, for a candidate to get the notion that he is “the бу 
person in all the world able to fill this job,” even though, 10 А 
certain very real sense, this тау be true. Human experience 
is more in accord with the sentiment that there are many 
people who could do a given job; and conversely, that for every 
person there are many jobs in which he might do at least mink 
mally well. In view of the fact that we know so little abou* 
the functioning of traits, let alone their organization, caus л 
tion, and growth, it seems safe to say that the summation 9 
the weighted scores method, perhaps in the general case p 
closer to the realities of *hu Log 


man experience" than most 0 


22 Or a similar formula in terms of В. 
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other methods cited, with the exception that when one is look- 
ing for unusual talent (with the expectation of hiring only one 
or two individuals at most from “among all the world") then 
the predominant or outstanding merit method is obviously 
superior. When employing the latter method if one is going 
to hire a very large proportion of all of the available candidates, 
then clearly one wants to be sure to refrain from getting any 
large aggregate of “weaknesses” and the minimum trait method 
is good. In this case it is understood that it is the production 
of the group rather than of a particular employee which deter- 
mines the personnel bank-balance of the selector. If one is 
going to hire quite a few candidates but still a very limited 
number compared with the total number of applicants, the suc- 
cessive hurdles method will be economical and withal effective, 
Providing the method is carried out in the approved fashion 
outlined above; namely, the administration of the most valid 
trait first, then of the next most valid, and so on down to the 
least valid last. When using this method Ruml’s rank tan- 
gential coefficient? may be of aid in establishing the “critical 
Scores.” (Critical scores are in none too good repute among 
Psychologists today. The concept grew up in a day when it 
Was believed that failures were due to one "type" of humanity 
and successes to another, and that accordingly, if only one built 
the right kind of a test, he would secure a bimodal distribution. 
If only this were not untrue, it would be a very definite concept 
in view of the fact that it would be easy, at least comparatively, 
to determine what point should be set in order that the over- 
lapping of distribution A on B, should be a minimum.) Where 
the numbers of cards to be sorted is “small” and the categories 
of traits are few in number, Keysort or Speedsort has several 
advantages, namely, that (1) a sorting needle is the only para- 
Phernalia required and (2) some capital presumably may be 
made of the visual channels on the edges of the cards as suc- 
cessive sorts—preferably in a descending order of the weights 
Ог importances of the traits—are made. In this latter case an 
ee 

273 Ruml, Beardsley. “The Reliability of Mental Tests in the Division of 


E Academic Group." Psychological Monographs, Vol. 24, No. 4, Whole No. 
5, 1917, р. 59 ff. 
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unbroken card across channels, the sorts being nearly com- 
pleted, would indicate the case of a person below the minimum 
(how much can quickly be inspected on the card in question) 
on the category in question under observation. 

There probably are other methods of selection. If so, it 18 
hoped that this paper may encourage others to discuss the re- 
maining methods and to compare their advantages and short- 
comings with the methods herein outlined. Selection through- 
out the world—particularly in areas where so much leadership 
personnel has been destroyed by war—will be more important 
in the days to come than in any recent historical period. 
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THE COUNSELING PROGRAM OF THE VETERANS 
ADMINISTRATION 


CARLOS E. WARD ax» GWENDOLEN SCHNEIDLER 
Vocational Rehabilitation and Education Service, 


Veterans Administration 

Tue Veterans Administration is preparing to make the ser- 
Vices of qualified counselors available to veterans throughout 
the nation. Counseling is being offered to all veterans who 
are eligible for vocational rehabilitation under Public Law 16, 
78th Congress, or for education or training under Title II of 
Public Law 346, 78th Congress, and it is evident that a very 
large percentage of the veterans of World War II will be eligible 
for the benefits of these two laws which are being administered 
by the Veterans Administration. 

Public Law 16, 78th Congress, approved March 28, 1943, 
may be regarded as a Disabled Veterans Vocational Rehabili- 
tation Act, since its principal purpose is to provide vocational 
rehabilitation to overcome the handicap of disabilities which 
Were incurred as a result of service in the armed forces during 
the period from September 16, 1940, to the end of the present 
War. Vocational rehabilitation will usually be attained through 
training provided for each veteran to fit him for employment 
Consistent with the degree of disablement and suitable to re- 
Store his employability. 

Public Law 346, 78th Congress, approved June 22, 1944, 
the correct title of which is "The Servicemen's Readjustment 
Act of 1944,” became so widely known as the *G. I. Bill of 
Rights" prior to its enactment by Congress that many people 
still speak of it as the “G. I. Bill." Title II of this Act provides 
that veterans who meet certain eligibility requirements and 
Whose education was impeded, delayed, interrupted, or inter- 
fered with by reason of entrance into service, or who desire a 
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refresher or retraining course, may be given education or train- 
ing courses at approved institutions of their choice within cer- 
tain time limitations prescribed by the Act. The veteran’s 
length of service is an important factor in determining the 
duration of the education or training to which he may be 
entitled. Comparatively few veterans of World War II would 
fail to qualify for at least one year’s education or training under 
Title II, if they should choose to apply. 

When the fact is considered that complete counseling ser- 
vice, including modern techniques of testing and the use of 
compendia of systematized occupational information, is abso- 
lutely essential to the selection of an employment objective and 
of a training course suitable to the disabled veteran in need of 
vocational rehabilitation, it is evident that this alone would 
require an extensive Veterans Administration counseling pro- 
gram. But when to this is added the Veterans Administra- 
tion's responsibility for counseling all eligible veterans who 
request educational and vocational guidance in connection with 
their applications for education or training under Title II of 
The Servicemen's Readjustment Act, then the magnitude of 
the task which this agency faces takes on new proportions. 

While “educational and vocational guidance" is the term 
used in Public Law 346 to designate this service, the Veterans 
Administration has developed its counseling program with most 
careful regard to the fact that veterans will have greater assur- 
ance of achieving their educational objectives or occupationa 
goals when mental conflicts, emotional maladjustments, ап 
other types of personal problems are alleviated prior to °" 
parallel with counseling and training. The plans and pro 


cedures of the Veterans Administration, therefore, are aimed at . 


providing such thorough and complete counseling to each vet- 
eran claimant that he may be assisted in making the adjust- 
ments necessary for a useful life as a citizen. For this counse" 
ing service it is most important that competent, profession y 
trained, and otherwise well-qualified persons be employed- 

For many veterans, guidance in the selection of an OCC" 
pation or of an educational objective may be all that will be 
required. Other veterans may need assistance in handlin& 
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personal problems, in resolving mental conflicts, in attaining 
emotional stability, or in learning how to secure and hold em- 
ployment. The Veterans Administration plans to furnish all 
types of counseling service which may be required in the case 
of an individual veteran. The veteran will also be guided in 
making intelligent use of other clinical and professional services, 
available to him through the Veterans Administration and 
other agencies, for the purpose of assisting him in making and 
maintaining the mental, emotional, and social adjustment 
essential to the attainment of his objectives. Each veteran is 
counseled in accordance with his needs as a person and educa- 
tional and vocational guidance are not given without refer- 
ence to the consideration of the other problems which affect 
the life of the individual. 

'The field organization of the Veterans Administration has 
been extended to carry out the general policies, plans, and pro- 
cedures developed in the Central Office for counseling vet- 
егапѕ. The program was started in 1943 by placing Vocational 
Advisers in the 53 regional offices in the different states. As 
the number of veteran claimants increase, Veterans Adminis- 
tration Guidance Centers are being established in a number of 
Cooperating colleges, universities, and other educational insti- 
tutions which have personnel qualified to render counseling 
Services. This plan makes the counseling service accessible to 
Veterans at points nearer their homes; it provides for the par- 
ticipation in this counseling service of a number of well-quali- 
fied people in colleges and universities who would not wish to 
Sever their relationship with their educational institutions but 
Who are able to contribute highly valuable service in the coun- 
Seling of veterans; it affords veterans thorough and complete 
Counseling in a suitable setting; and it enables the Veterans 
Administration to make arrangements, in advance, with educa- 
tional institutions which, because they are going concerns, will 
be able to start counseling service to veterans on comparatively 
Short notice when an increase in the number of claimants makes 
it desirable to have a larger number of Guidance Centers in 
any regional territory. 

The fact that a veteran is requested to report for counseling 
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at a Veterans Administration Guidance Center located at any 
particular educational institution places no obligation upon 
him to take educational or training courses at that institution. 
Each Veterans Administration Guidance Center provides 
counseling service to any eligible veteran who may apply if 
he resides within the area to be served by the Guidance Center, 
and each veteran may take his training in any approved edu- 
cational institution or training establishment where he can 
secure courses appropriate to the attainment of the educa- 
tional or occupational objectives he has selected. 

One of the most important advantages of the plan for estab- 
lishing Guidance Centers in educational institutions is the pro- 
vision it makes for a continuing supply of trained counselors, 
so that when the time comes to expand the service still further 
and to extend it from the nuclear points at the colleges and 
universities to the communities more remote from these it may 
be possible to make this expansion without reducing in any way 
the quality of the counseling service. Considering the prob- 
ability that before long it may be necessary to provide coun- 
seling service at some 400 Guidance Centers at educational 
institutions and then to prepare to expand the service still 
more by supplying trained counseling personnel for additional 
offices of the Veterans Administration, it becomes apparent how 
important it is to make provision for a practical program of 
counselor training. 

For this purpose the Guidance Center plan which locates 
the practical work of counseling in the educational institutions 
which provide counselor training is in many ways ideal. Such 
colleges and universities not only are able to train counselors 
through their usual classroom instruction, but they are also 1” 
a position to combine with this instruction observation 0 
actual counseling procedure. As the new counselors develoP 
they can be given closely supervised practice in counseling tech- 
niques. Thus the established Guidance Centers not only Pt 
vide veterans with the services of well-qualified and exper" 
enced counselors, but they also serve to increase the number 0 
trained counseling personnel. à 

At each Guidance Center the Veterans Administration will 
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have at least one Vocational Adviser and one Training Officer. 
The members of the faculty of the educational institution who 
are assigned to counsel veterans at Guidance Centers are called 
“Vocational Appraisers” in order to distinguish them, in the 
records, from the Veterans Administration Vocational Advisers. 

The Veterans Administration Vocational Advisers are se- 
lected from lists of professional counseling personnel prepared 
by the U. S. Civil Service Commission, and the counselors who 
are employed by the educational institutions to render service 
to veterans in Veterans Administration Guidance Centers have 
similar professional qualifications. The success of the counsel- 
ing program depends upon securing counselors with the requi- 
site professional training, who are technically qualified in psy- 
chology and in tests and measurements, who are experienced 
in performing counseling functions and who can work effec- 
tively in an organization following prescribed procedures. 

The counseling principles, methods, procedures, and tech- 
niques used in the Veterans Administration counseling program 
are contained in the “Manual of Advisement and Guidance” 
which is being printed by the Government Printing Office, and 
these will not be described here. Short, intensive training 
conferences covering the specific procedures and techniques 
which are described in the manual will be conducted at selected 
locations from time to time as part of the in-service training 
Program for those entering the Veterans Administration coun- 
Seling program in order to supplement the basic background 
of training and experience which the adviser of veterans should 
have acquired over a period of years. 

The counseling program of the Veterans Administration 
both in the regional offices and in the Veterans Administration 
Guidance Centers adheres to the policy that counseling does 
hot meet required standards if it merely informs veterans of 
Occupational and training opportunities without making a care- 
ful survey and a thorough analysis of the individual’s educa- 
tion, work history, abilities, aptitudes, interests, and personality 
traits. In such an individual survey and analysis, extensive 
use of objective tests is required in most cases. It has been 
the policy of the Veterans Administration to administer only 
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those tests which the counselor prescribes for each veteran, 
rather than to make use of a standard battery for all. Coun- 
selors in Regional Offices and in Veterans Administration Guid- 
ance Centers select from well-standardized tests those which 
they desire to use and their selections are submitted to Central 
Office for review and approval. This plan has enabled the 
Veterans Administration, while initiating the counseling pro- 
gram, to utilize the best skills of counselors whose experiences 
in the administration of tests have been quite different, without 
waiting to train them all in administering a specified list of 
tests. As the program goes forward and as experience in the 
testing of veterans in connection with counseling is studied and 
evaluated, it is probable that the tests which serve a specific 
Purpose in a large number of cases will be more and more 
widely used and the selection will thus tend to become more 
standardized. ‚ 
To take a long-time view, the Veterans Administration’s 
policy of providing the services of professionally trained coun- 
selors for veterans should increase the number of persons who 
are trained for counseling service so that, after the present 
emergency is past, such counselors may be available to carry 
on more extensive student personnel programs in universities, 
colleges, and secondary schools. It should also enable com- 
munity agencies to render more counseling services to adults. 
A greater understanding both of the values and of the limita- 
tions of tests and of other counseling techniques may be 
attained through experience in this program. Another result 
hich may be expected will be an increased demand for coun- 
sema Services and for Professional training for counselors. 
This training will probably take a more practical form by 
including greater opportunity for clinical in-service training 45 
well as for theoretical work in psychology and related fields- 
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PERSONALITY TRAITS ASSOCIATED WITH 
ABILITIES. I. WITH INTELLIGENCE 
AND DRAWING ABILITY 


RAYMOND B. CATTELL 


University of Illinois 
I. Conception of the Research Problem 


In is customary to think of abilities as powers having an 
existence independent of other personality traits, ie., of dy- 
namic and temperament traits. In mathematical terms this 
18 founded in the conception of unitary traits as factors (6). 
Since even different ability factors are mathematically inde- 
Pendent of one another, it is not surprising that factors of differ- 
ent modality, e.g., temperament and ability traits, are still 
More confidently expected to be independent. Correspond- 
Ingly, in clinical and general psychological terms the usual 
Approach conceives of functionally independent traits or powers. 
Abilities, for example, are the tools of dynamic traits and may 

© used interchangeably by the same or different drives. Pre- 
diction of, say, the outcome of a son’s antipathy to his father 
°F a girl’s overcompensatory concentration on school subjects 
Tests first on an estimate of the strength of the drive but also on 
knowledge of the endowment in the various abilities which it 
may use. 
, The purpose of this paper is to show that the above analysis 
Is only a first approximation to the truth. It proposes a more 
refined conceptualization and presents some new data, which, 
together with the data of an ensuing article (5), constitute a 
slight initial foundation for a factual edifice in this realm. 


IL The General Nature of Ability-Personality 
Trait Connections 
oft Clinically the connection of abilities with dynamic traits is 
en quite striking. Inferiority overcompensations sometimes 
131 
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produce astonishing performances—relative to the individual’s 
LQ.—either in all school subjects ог in some obscure field in 
which the individual finds he can at once jump ahead of, and 
avoid outright competition with, his rivals (1). The writer 
has also known two or three “idiots savants” whose outstanding 
“trick” performance in computation or memorizing could be 
traced to some initial accidental display, of comparatively slight 
eminence, by which the individual discovered that he could step 
into the limelight from the drabness of institutional life. 
Psychoanalysis provides abundant instances of special skills, 
perceptual and motor, developed, like symptoms, out of the un- 
conscious drives, relentlessly seeking expression. The comple- 
mentary phenomenon of conscious drives, as sentiments, shap- 
ing abilities is so commonly realized as scarcely to demand 
illustration. Many of the special abilities distinct from intelli- 
gence isolated by research, e.g., Thurstone's Primary Abilities, 
may prove to be environmentally, dynamically shaped patterns, 
from general ability being impressed by particular investments 
of time and energy in certain conventional patterns of skills. 
(This aspect has been discussed more fully elsewhere (4) in 
connection with the theory of fluid and crystallized abilities.) 
The influence of major, overt sentiments upon ability patterns 
probably reaches its widest expression in respect to the self- 
regarding sentiment and it may be, for example, that the lesser 
mechanical aptitude of girls, and even the failure to find а 
mechanical aptitude factor among measurements on girls, will 
in the end be traced, not to difference of natural capacities 25 
such, but to differences of dynamic adaptation stereotypes 0 
the self-regarding sentiment. 
Naturally, causal connections can run in either direction, ОГ 
in a “causal circle"; and from general psychological observation 
a clinician would confidently say that examples of these three 
theoretical possibilities exist with almost equal frequency. In- 
terests produce discriminatory and motor abilities as discussed 
above. But the individual who finds himself endowed with 
certain good natural abilities is likely to enjoy exercising them 
and, in a competitive world, to find the dynamic pattern of his 
self-regard increasingly shaped by these abilities. Conse 
quently, the establishing of connections between abilities ар 
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personality traits is only the first—though a highly necessary 
—stage in investigation, needing to be followed by exploration 
for causal connections. 


Ill. Conditional and Wholistic Factors 


To dissect the present research problem into its ultimate 
Toots one would need to ask whether the very distinction be- 
tween ability, temperament, and dynamic traits may not bea 
chimera. Fortunately we have no need to enter upon so oner- 
ous an undertaking here, having carried it out elsewhere (7) 
and emerged with the conclusion that these common sense dis- 
tnctions are real enough and capable of operational definition. 

The psychologist familiar with correlation procedures is 
More likely to ask the very matter-of-fact question: “If abili- 
ties and non-cognitive personality traits sometimes correlate 
appreciably, how is it that the factors which have so far been 
discovered have been either pure ability factors, pure tempera- 
ment factors, or pure dynamic factors?” The answer is, first, 
that there has been a tacit or unconscious conspiracy to main- 
tain certain influences constant when giving tests, without men- 
tioning—often without realizing—that such artificial conditions 

ave been set up. We correlate ability tests given under con- 
ine of quiet, of concentration, of common intention to do 
shoni est. We correlate emotional responses in, say, nursery 
abili children, observed under conditions in which cognitive 
ur les аге not required in order to manifest emotion. That 
«di Say, we always hold constant, in the measurement situ- 
on, the personality manifestations in which we are not inter- 
ested, 
жын obtained under such test conditions we have named 
Я бара oe Since everyday life provides such constant 
are by Ons in a fair proportion of situations, conditional factors 
of Белер means useless artificialities, when one comes to tasks 
ical prediction. 

Ут conf (пайт fo Чест патоп 

Sample of the s confine : : zs 
ге look; possible range of personality performances. ey 
i ks нас for abilities, or even musical abilities only, or even 
judge pitch alone. A very narrow range means less 
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communality, more unrelated specifics, fewer factors, and larger 
emphasis on the factor peculiar to that region. 

The hypothesis can be put forward, therefore, that if one 
measured a collection of performances each under the naturally 
existing, varied conditions usually associated with the perform- 
ances, and if one took an extremely wide sample of aspects of 
personality, many of the factors emerging therefrom would 
range at once over abilities, temperament traits, and dynamic 
traits. Factors obtained under such conditions we have called 
wholistic factors. Conditional factors will then appear as trun- 
cated forms of corresponding wholistic factors, cut down to 
their manifestations in a single modality. 

General psychological considerations support such expecta- 
tions. One can readily think of both constitutional and en- 
vironmental mold source traits which should appear as wholistic 
factors. A single gene might be responsible for modifications 
in, say, certain parts of the midbrain, the cortex, the pituitary 
and the inner ear, causing simultaneous endowment in certain 
dynamic needs, some motor capacities, specific temperament 
traits, and particular auditory acuities. Similarly, particular 
school systems might frequently produce environment mol 
factors of covariation in certain acquired abilities, in forms of 
inhibition, and in specific interests and habits. Ё 

The exploration described below promised for the first time 
to reveal such wholistic factors if they exist; for it took a de- 
liberately very wide array of personality aspects; in fact 1t 
sampled from what has been described in more detail elsewhere 
(2) as the personality sphere. Specifically it dealt with 3 
clusters of traits, representative of all traits in the dictionary» 
rated for 208 adult males who had also been tested for intelli- 
gence, mechanical aptitude, drawing ability, verbal ability, 
mathematical ability, etc. The details have been describe 
(3). 


IV. Personality Associates of Intelligence 


The principal personality correlate of intelligence know” n 
psychologists is moral character and it is with respect to this 
correlation that the vast majority of past data has beer 
gathered. The correlation is established through studies A 
intelligence of delinquents, alcoholics, etc., contrasted with “4 

LI 
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mal controls, and through correlations of intelligence with rated 
or measured moral character qualities in a “normal” population. 
This whole field has been very systematically and compe- 
tently surveyed and summarized by Chassell (11). The nor- 
mal-delinquent and other dichotomous data on the one hand 
E ie essere data on the other, when expressed in cor- 
s ing correlation form, agree very well, as also do the data 
И есу and those from such actual conduct studies as 
= ated artshorne, May, and Maller (14). Chassell (11) 
E са the results from many researches and several thou- 
the di subjects as follows: *Expressed in correlational terms, 
tained relation may therefore usually be expected to fall 
€tween .10 and .39, and the true relation to be under .50.” 
Es most of these researches are on groups of restricted range 
sell Em for the population as a whole would be higher. Chas- 
iud zu ly estimates that the correlation between intelligence 
К БЫ moral character qualities of self-control, unselfishness, 
E | ity, industry, loyalty, etc., when corrected for narrow 
ation and attenuation, is close to .60. There are clear indi- 
Жа 5 that the correlation is somewhat higher for children 
Чон ог adults and, curiously enough, higher for the correla- 
with intelligence than for correlation with school achieve- 


me : з : : 
ee This last, together with the fact that measured intelli- 
M Се apparently correlates as well as rated intelligence, reduces 


chilay ent of the possible criticism that in some rating (teacher- 
Sas Situations the “goodness of character” might be a spuri- 

Product of pleasing authority by doing good school work. 
е findings of the present research may be viewed from 


|, two 
re] angles (1) as straight correlations viewed in terms of cor- 


ati А р 
^ 9n clusters or surface traits and (2) in terms of factors or 
Tce traits, 
take не intelligence, in this group of adults, was found to 
5 place in two clusters, as follows: 


Ch 
(tems in do, Indexed as B3 in (2) Cluster Indexed as B4 in (2) 
. .SOscending order of correlation (Items in descending order of correlation 


So) 


Original үн Temaining items) with remaining items) 
. Banal Intelligent .........- Ni e 
ee Clear thinking ...... V. .. Incoherent 

. Interests Logical ability SS ws, 
narrow Given to reasoning .. V. 
V. ... Emotionally ^ Clever ...........-- v. 
dependent Spatial-visual ability . v. 
++» Quitting Mathematical ability . v. 
. Unintelligent ^ Analytical .......... v. 
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In these clusters every variable correlates with every other 
above .40 and the mean r is about .75. Apart from various 
“cluster appendages” to these nuclei, provided by traits which 
correlate with some but not all of the items, these are the only 
two clusters into which intelligence enters. B4 is essentially 
intelligence manifested as a cognitive ability surface trait. B3, 
which has also been found in the Sanford-Murray research, 
shows, on the other hand, intelligence amidst its closest person- 
ality associates. 

2. In the factor analysis of the 35 clusters taken to repre- 
sent all aspects of personality, one of the twelve factors—the 
second largest! in its contribution to the variance—appears te 
be a factor of general intelligence. Its loadings for all variables 
may be read in the published table (3). Here we shall discuss 
only traits with outstanding loadings. The six traits (each 2 
cluster) with highest loadings are as follows: 


Loadings in Personality Factor B? 


(2) .52 (Intelligent) ............ v. .... (Stupid) 


(Clear thinking) .. . v. .... (Incoherent, confused) 
(Clever) ......... "E 


(11) 47 (Persevering) ... у. .... (Quitting) 
(Painstaking) ... v. .... (Slipshod) 
(Conscientious) . у. .... (Conscienceless 

(4) 43 (Thoughtful) ... v. .... (Unreflective) 
(Deliberate) .. . у. .... (Impulsive) 
(Nusteke) sesine sureci v. .... (Profligate) 

(28) 42 (Stable emotionally ..... у. .... (Changeable) 
(Self-respecting) Mi. ges 
(Self-controlled) v. .... (Unselfcontrolled) 

(12) .41 (Intellectual) eee 
(Analytical) .... v. .... (Unreflective) 

(Wide interests) v. .... (Narrow interests) à 
(3) 41 (Independent) у. |... (Emotionally dependen 

(Reliable) .. es 0v. .... (Undependable) 

(Mature) азгы... v. .... (Emotionally immatures 


irresponsible) 


ces 115 
In the f rotation variable 28 falls slightly and 29 takes 1t 
place in the first six. 29 is 


Alert .. Чэбдик upset Absent-minded 
Energetic v. ... Languid 
Quick Жы „экеа айка Slow -— 
"m š aT: å m 
ite remains in this position, and substantially unchanged in characters 
alternative « and В factorizations which have been offered (10). een 


B - + A САЯ E has b A 
? This factor, which is well known as Spearman's “д” in ability studies, Ваз ality 
called “В” in the personality realm, to conform with the series of general Р, e іп that 
factors and to express by alphabetical order its relative order of magnit? 
series. 
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| Тһе numbers at the left identify the cluster variables in the 
original list recorded in the factor analysis (3). When an 
actual intelligence test was correlated with the personality vari- 
| ables it confirmed the factor as being the ability factor of gen- 
| eral mental capacity, outcropping in the personality realm, for 
It gave the following highest correlations, which agree substan- 


tially (5 of the 6 being the same) with those found in the factor 
analysis. 


(2) Intelligent, analytical, etc. 
(12) Intellectual interests, etc. . 
(11) Strong-willed, conscientious .... 
(29) Psychophysically vigorous, alert .. 

(3) Wise, mature, polished ........... 
(35) Smart, assertive «eee 


| Опе may next attempt to glean the evidence resident in past 
r » М . . . 

| es as to the correlation between intelligence and indi- 

| idual traits in the realm of character—as distinct from total 


Character—to see if the above order is confirmed. Below we 
ave taken the researches of Terman, Webb, and others re- 
| е by Chassell and averaged the 7’s by a method roughly 
| уна. the result for different samples and for attenuated 
соза However, the method does not justify recording 
enl: correlation magnitudes and we merely give the rank 

of the traits, the 7's of which range from about .7 to .3. 


(V Cooperativeness (Highest) ......:4: —€— (Average of 7 75) 
ў eliability, trustworthiness, responsibility ... (“17 id 
й ndustriousness (mainly іп school) ..-+----- gi ^) 
Sonscientiousness —— "a 10 = 

арау ааа ev 22 т 2 sl 

oral habits and ideals . \ s б m 

Mselfishness ........... eC н 2 “ ) 

Су (Dowest) муше зә. анна € * 25) 


ч. some remaining evidence is presented we shall defer 
ings n as to whether this order agrees with that of the load- 
the p ма В factor. The remaining evidence of the nature of 

| B E actor is to be obtained from looking into the traits which 
| ес actor does not load, i.e., those in the hyperplane of the 
Py oa. го is a necessary enquiry in the recognition of any 
d Or cw | he B factor had presented some difficulties in rotating 

| and e € structure because it produced only a relatively faint 
certain hyperplane: there were relatively few variables 
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which it did not influence. (Only the D factor exceeded it in 
this width of influence.) The personality variables thus inde- 
pendent of the B factor (loaded 0.05 or less, in both a and P 


rotations) are: 
(5) Hypochondriacal 


Selt-deceiving .........- e Mirco css Realistic 
Nervous and neurotic 
(7) Extra punitive ............. Su сыа 
Headstrong 4:6 ns у. ....... Gentle tempered 
Exhibitionist заса 99 wes WU osma Self-effacing 
(QI) Bonu ipren Y isses Modest 
JÁSSeItIVE. ез» рда MS уюша» Submissive 
Conceited чоокыр rivis у. Self-critical 
(Oy. “Whale eas) „неликтен у. Grateful 
Hardheartéd уоона v. Softhearted 
Short-tempered ............. NB. aeneae Easygoing 
(18) High-strung ............... YE aoas Unexcitable 
Hurried Vi жыруы Lethargic 
Vivacious . Xi yx Bi 
(24) Sadistic Mi. sies аа Not sadistic 
Suspicious NE spun uf Trustful 
Mulish ... Ж. ышкан Reasonable 
(23) Eloquent .. Mix cs laetos Inarticulate А 
Flattering „рК Natural, self-effacing 
(32) \Optimistic: «s saxessacamcesn AE аллана Pessimistic 
БИЕШ; атаана аа ЕТА Worrying 
Inspection of the correlations of traits with the actual zh 
telligence test confirms the independence of these traits Wo 
respect to general ability, and adds, within the 0.05 limit, t 


e: 


traits numbered 6, 14, 16, 19, 22, 26 and 33. Briefly these 27 
Softhearted, Antisocial schizoid, Active neurotic, Friendly 
frank, Aloof, Hypomanic emotional, and Introspective hurries 
respectively. 

If we study the factor loadings of the single variable id 
gent” (No. 2), which has the highest loadings of апу variab e 
in the factor of general ability (B), we find that it has negligib 


«[ntelli- 


loadings in any other factor. Its possibly significant loadinE* 
all below .26, are in the factors G, J, and K—factors of charact 
become 


and education which, in the alternative В factorization, i 
absorbed in the B factor. In short, the direct personality ane 
ciations of intelligence are wholly accounted for by the B facto 
loadings. : 
However, the B factor as a whole has some correlations wit 
other factors, for we rotated for simple structure, eve? ү i 
attaining it meant leaving some factors somewhat out of 2 
thogonal positions. Notably, this factor correlates positive 
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with C (.32) and, less certainly, with D, С, and J. When 
second-order factors (9, 17) are calculated from the correla- 
tions between the primary personality factors one, and possibly 
two, second-order factors are found, loading factors B and C 
Positively and more highly than any others. 

Adequate discussion of the meaning to be given to the novel 
concept of second-order factors is not possible here. (See 9 
and 17.) The writer believes that the most likely explanation 
of the second-order factor loading B and C (Emotional Sta- 
bility, inverse of General Emotionality) is that it is a social 
Status factor, expressing the genetic adhesion (8) produced by 
assortative mating between intelligence, emotional stability, 
and other “success”-generating qualities that are intrinsically 
distinct, 

_ So much for the sheer description of the connections between 

Intelligence and personality structure. A fuller discussion of 

the Possible interpretations and origins of these connections will 
€ taken up towards the end of this article. 


V. Personality Associates of Drawing Ability 


About the personality associates of creative artistic ability, 
notably about the alleged “artistic temperament,” much has 
€en written outside scientific psychology. One could wish that 
Our data might throw light on this matter, but actually it is 
Testricted to drawing ability and may or may not have refer- 
ence to total artistic feeling and creativity. One hundred and 
twenty-eight subjects, selected from the 208 adult males in the 
total Personality research for more uniformity of age and edu- 
С Чопа] background, were asked to draw a man sitting ш a 
ited reading a book. The drawings, all done within a 20-min- 
bud Period, Showed marked variations in resource, originality, 
dig it but were rated, by art teachers, for зү ж ewe | 
о у^ The reliability of the artistic ratings, as between the 
Judges, was .88. 
9rrelations with personality variables were carried out 


Separately within each of the 8 groups of 16 within which the 


en " ERE E 
к ere rated. The correlations agreed in sign in 6, 7, or all 


8 tq 
2 : » 
Correctness of drawing, proportion, quality of line, expressiveness. 
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of the 8 groups for the following variables: 9 (—), 11 (+), 15 
(+), 21 (+), 22 (+), 23 (+), 25 (+), 28 (-), 29 (+), 30 (+), 
31 (+), 32 (+), 33 (+), 34 (+) (see variable list in 3). On a 
population of this size any correlation above about .22 is sig- 
nificant at the 1% level. The following mean correlations (for 
8 groups of 16) are therefore worthy of especial note from 
among those listed above. 


Drawing Ability with: 


БШ РАЛУ A КЕЕ И. арад Habit- Оша 
DtUlEIVe. „енка Ж жор ogica r=.29 
(34) | Careless of material 
tlinpS ЖТТ Vi: рига Thrifty 
Sociable) sss +з уна» Ure Ve ross Shy = 29 
(31) RESPONSIVE: ананга Mi xus iuis Aloof r=.2 
|+ “днн AAR Бр, сараа Quiet 
Incontimnt ............. Ve. wa ree Inhibited 7 
(30) Gluttonous: «veniente m Queasy ‚=? 
О ааа оналар We viria Unenquiring 
Gratefül „>ле унге» Vir жасу» Thankless = 6 
(9) Softhearted огуз secos Vs. senses Hardhearted =? 
Easygoing у. . . Short-tempered 
Tough .. v.. . Sensitive = 25 
(33) Lethargic a A. аде Hurried кеч 
Talkative RE Introspective 
Pap AES Ns uvas Absent-minded 4 
(29) | Energetic-spirited......... Yi suns Languid gem 
QuicE 5 „неее siemens ‚Жа Slow 
Energetic-spirited ........ OE: сз Languid = 24 
(21) Self-confident ........... BIS. casae in Self-distrusting fien 
Debonnaire ............. Ж. areis Aiken sEO veo eta 
| Responsive .............. VE weoma s Aloof 222 
(22) ERE E чесе ы serat Cold-hearted i 
Social interests .......... 5 NEA Brooding 


Because of the consistency of the correlation in all eight 
groups, despite its lowness, we suspect systematic connection? 
also with (23) Exhibitionist, eloquent, flattering, (13) Easily 
jealous, self-pitying and (32) Optimistic. 

The agreement of this syndrome with the popular 
type of the earthy, gay, spirited, unstable, careless, Boh he 
artist is very striking, yet it must be emphasized that at t " 
time the ratings were made no one had any idea that the D. 
pose was to correlate them with artistic ability. The corre Ж 
tions are low, but there is a reason additional to the simple m. 
tistical test of significance for believing that they are real. mae 
is that the pattern of traits corresponds strikingly to ? us á 
tional unity, or small group of functional unities, already fou 


stereo" 
emian 
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in the factor analysis of personality (3). Seven of the above 
eleven variables occur among the small group of variables 
highly loaded with and used for defining the F factor of Sur- 
gency .. у... Melancholic Desurgency. Viewed more closely 
(see table of loadings, p. 88 of 3), this cluster of drawing ability 
personality associates is seen to be systematically produced by 
the F factor (Surgency) in collaboration with H (Charitable 
hathymia .. у... Obstructive Schizothymia) and to a lesser 
but perceptible extent with Æ (Dominance) and J (Vigorous, 
Sessional” Character). That is to say, if one picked out 
the variables high in these four factors, neglecting the influence 
of all other personality factors, and if one gave predominance 
to Fa slighter role to И, and a dash of E and J, the correla- 
ions between the members of this cluster would be fully ac- 
Counted for. At this stage of research, however, no systematic, 
Sxact partialling out of the correlations has been attempted. 
Ne interpretation of this finding is discussed more specula- 
tively below, 


VI. Interpretation of Correlations with Intelligence 


The correlating of intelligence with a complete range of 
Personality manifestations confirms the earlier impression of 
narrower experiments that it correlates almost exclusively with 
m may be called "character" qualities and that the сей 
зы. are of substantial magnitude. Moreover, among the 
eee qualities themselves the traits persevering, consci- 

Ous, self-controlled, reliable, emotionally independent, in- 

Nstrious, etc., correlate somewhat more highly than unselfish, 
Motionally stable, sincere. It looks as if intelligence is directly 

01е associated with character conceived in a narrow, self- 
oe sense, and with respect to habits that are acquired 
ofc through conscious ideals, rather than with basic emo- 
such. "fegration and goodness of character in the wider sense 
from as might result from the emotional adjustment yp 
tively, © upbringing of the first few years or from some rela- 
also ce nstitutional stability. The above character d 
tive X Monstrably include breadth of interests, habits of reflec- 


ought, and analytical habits of approach to problems. 
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The association with the above restricted realm of character 
qualities may be interpreted either as the functioning of a single 
factor В (В factorization), or due to three further factors, С, J, 
and К, which are themselves correlated with B as factors (а 
factorization). The pros and cons of the alternative rotations 
which give these two possible interpretations have been dis- 
cussed elsewhere (10), and until further research is done a 
choice is impossible. However, no matter which of these 
analyses is accepted, the present writer considers that the gen- 
eral psychological knowledge in this field favors the hypothesis 
that the character correlations are due to better intelligence 
leading to better learning, which would be expected to show 
itself in conduct situations almost as much as in academic situa- 
tions. That is, the character patterns are “environmental 
mold” traits, patterns of reward and punishment in the culture; 
which “take” better on a basis of good constitutional “д” еп- 
dowment* than on poorer soil. At least during childhood it 
is an intelligent adaptation to develop good character. The 
converse hypothesis, that good character qualities lead to better 
performance in intelligence tests, can be given only a negligible 
role, in view, for example, of the negligible effect which conatiVe 
variations of all kinds have been shown to have on intelligence 
test performance. 

Although character qualities of the above restricted kind are 
directly to be connected with intelligence, the character quali- 
ties (for such they are commonly considered) of the C factor; 
which we may call deeper emotional integration or stability 
(see full description in 3), are connected only indirectly; bye 
second-order factor, saturating both C and B, and causing thelr 
appreciable correlation. We are inclined to think that m 
second-order factors will turn out to be best explained in caus? 
terms as a common cause influencing both the correlated var 
ables (i.e., as the third of the logically possible alternatives n 
explaining correlation). 

4 That these correlations run a little lower with adults may be due to ШЕ s 
that years of trial and error experience compensate and reduce the learning gains. 


intelli то теси А г, even 
to intelligence alone. After all, much conduct learning is blind trial and его 


У H i igent 
for the more intelligent. A radically different alternative would be that ja eld 
adults more quickly unlearn the moral habits taught them as children, but *Jesirable! 


assume that intelligent people more frequently consider moral habits um 
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The C and B connection arises, we believe, from the effect 
of family and of social status (a) environmentally, in that more 
intelligent children, having more intelligent parents, will tend to 
get wiser handling of those emotional problems of the early 
years of childhood which determine the deeper emotional ad- 
Justment (c) and (b) genetically, by reason of what has been 
Called the social status orientation of genetics patterns (9) 
showing itself in socially conditioned genetic adhesions. In this 
Case, we presume, both high intelligence (positive B factor) and 
high emotional stability (positive C factor) are selected for 
Social promotion and become genetically linked by assortative 
Mating within social classes. 


VII. Interpretation of Drawing Ability Correlations 


Previous studies of drawing ability® as an artistic perform- 
ance, as distinct from Goodenough’s ingenious use of it to mea- 
Sure intelligence (14), have on the whole failed to give us any 
“lear picture of its ramifications as an ability or its connections 
With artistic abilities in general. Meier's survey (16) of the 
Problem stresses that emotional and temperamental qualifica- 
mons are as important as those connected with mere skills. 

lebout’s (18) research, unfortunately on few cases, found no 
“PParent superiority in various motor skills among those su- 
p nion in drawing, but found superior power of observation and 
abi ity to retain visual impressions, for days or for months. 

тер (12) findings point the same way, showing better mem- 
Ог visual form but not better motor ability, and indicating 
ап ter emotional sensitivity and more neurotic tendencies 

Опа abler artists than among others. 
ee Tüsts familiar with the life of outstanding e of w 

Eu Present were inclined to consider Van me de p 
iote use-Lautrec, or Whistler more typical tl an the Wes 
Dess о mod to impute higher general emotionality, s ui - 
labi: and ability to see everyday things freshly (through greater 

ty), and more imagination generally, to the true artist. 


ne i К B H x 
x ghe add the consideration that psychotics have some 


ij . 
the чалу Studies have unfortunately been on artistic ability as : Ше begging 
Оп as to whether several distinct talents may not be involved. 


Brea 
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times been known to paint for the first time after the onset of 
the psychosis, and that several instances are known, e.g. 
Churchill and Hitler, of vigorous minds turning to painting in 
times of political or professional failure. In short, the present 
writer would argue that any researcher in the future who makes 
statistical analysis of life in situ, i.e., of actual artists is likely 
to find the personality traits associated with artistic perform- 
ance, as defined by these more intensive researches, amply 
illustrated. 

Common observation also seems to indicate that there is a 
fairly powerful hereditary influence in drawing ability. On the 
bases of the factor associations found here we would argue con- 
tingently that it is largely a factor for the temperamental ten- 
dencies which is inherited and that the drawing ability per 5€ 
is acquired on the basis of the interests which these generate. 

The F factor of surgent temperament (associated in 1t$ 
extreme loadings with conversion hysteria), with its test mant- 
festations of “fluency” and “imagination” which have long 
since been demonstrated to be part of it (1), seems to be the 
first condition for the development of drawing ability. The 
cyclothyme-like factor H—which, incidentally, is one of the 
two factors loading “aesthetic interests” in the original factor 
analysis—is the second factor having correlation with drawing 
ability and perhaps corresponds to the element of more passive 
sensitivity and appreciation in the interests which lead to draw- 
ing skill. The connection of cyclothyme temperament Wit 
appreciation of color and the visual arts was stressed by 
Kretschmer (15), who considered the high hereditary incidence 
of cyclothyme constitution in the Alpine-Mediterranean racia 
regions to account for their pre-eminence in production an 
appreciation of visual art. The indicated correlations of 
(Dominance) and J (Vigorous, Obsessional Character) are to? 
slight to justify discussion until further explored. 

If clinical type observation, aided by the definiteness of 
factors engendered by factor analysis, may be admitted as 4 
guide to further research, the present writer would suggest that 
the following associations are likely to become important W 
total, creative artistic ability, as distinct from drawing ability 
alone, is studied. First, observation suggests that the loadings 


PERSONALITY TRAITS ASSOCIATED WITH ABILITIES 145 


of the cyclothyme factor (A or И) will be found almost as im- 
portant as those of Surgency (F). Secondly we can surely 
expect that factor C, in its negative loadings, will have some 
role. This factor connotes the instability, plasticity of thought 
and purpose, richness of emotionality and freshness of emo- 
tional approach, which the world evaluates one way and artists 
another. There is no hint of this factor in our results because 
we did not deal with creative artists but with soldiers selected 
for emotional stability and measured with respect to straight 
drawing ability. However, we suggest that it would be desira- 
ble to confirm the above personality factor associates of drawing 
ability before directing research to the more complex person- 
ality problem of artistic creativeness. 


VIII. Summary 
Intelligence. Intelligence appears as a general factor (В) 
among personality traits, loading particularly character traits, 
znd notably those good habits which may be consciously ac- 
quired. This factor correlates, however, to the extent of about 
3, with a distinct factor (С) of emotional stability and inte- 
8ration, and together with other factors they yield a second- 
order factor, which may be the genetic adhesion of intelligence 
= s етрегатепа! (emotional) stability produced by social 
ain cation. Past findings fit in well with these factorial 
Tpretations. 

of чайна Ability has significant correlations with eight m 
Sii. 35 surface traits used to represent the total personality 
re] T€. "These can be best explained as due to low positive cor- 
ations of drawing ability with Surgency (F factor) and 
sa Fme Cyclothymia (H factor), and possibly slighter cor- 
ons with Dominance (Æ factor) and Vigorous Character 
t ы юг]. This personality pattern very distinctly on 
Bust 2 Served in well-known artists, but it is xat таг 
alone artistic ability, as distinct from artistic drawing ability 
C poro likely to involve General Emotionality (Negative 
Xo Writer wishes to express his thanks to Thelma Alper and 
E Carvell for expert help in completing the drawing sec- 

n of this research. 
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FACTOR ANALYSIS OF OCCUPATIONAL APTITUDE 
TESTS 


Staff, Division of Occupational Analysis, War Manpower Commission 


OccupATIONAL research projects have been conducted by 
the Division of Occupational Analysis since 1934, when it was 
known as the Occupational Research Program of the United 

tates Employment Service. One of these projects has been 
the Construction of aptitude tests. Batteries of aptitude tests 
ave been validated for various occupations and are used in 
local offices of the Employment Service to aid interviewers in 
Selecting the most satisfactory beginners for referral to jobs or 
training courses. In developing an aptitude-test battery for 
an Occupation, a number of aptitude tests are “tried out” on 
employed workers (or trainees) for whom objective criterion 
ага can be obtained, and the best combination of tests con- 
Stitutes the battery. | 
he Division has recently been conducting a series of factor 
analysis studies to determine the important factors in its apti- 
ude tests, Interest in the problem of the basic factors or 
Undamental aptitudes underlying the Division’s occupational 
tests has been heightened by the recent emphasis on the coun- 
Seling approach in vocational placement and guidance pro- 
ome From a practical point of view, the preparation of 
Voti e Qd a дава 
can be countable task. owever, if О Ap A tae ciii 
tudes спава in terms of a few relative y in o e Hs 
Ences М be iege e a m "Ex of tests for 
genera] Job aptitude requirements, then ES ке: 
anal 2" Counseling purposes becomes feasible. Several factor 
late YSIS studies have therefore been conducted in order to iso- 
the basic aptitudes measured by the Division’s tests and 
а small number of measures of these factors for com- 
147 
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bination into a counseling battery. It is planned to standard- 
ize this battery for a large number of occupations which will 
then be classified into groups on the basis of similarity of apti- 
tude requirements. 


The Experiment 


Several experimental batteries of tests were administered 
to a total of 2,156 persons for the factor analysis studies. The 
data are divided into nine experimental groups. The number 
of subjects, the number of tests, the geographical location, and 
the factors identified for each group are shown in Table 1. The 
smallest battery consisted of 15 tests; the largest, of 29. There 
was a great deal of overlapping of the tests among the several 
batteries; but, in all, some 59 tests were subjected to analysis. 
Some of the factors found are well known and are similar to 
those which have been discussed by Kelley, Thurstone, and 
other investigators. Others have not received attention 
heretofore. | 

In group 0, 19 tests were administered to 1,079 male appli- 
cants for defense training courses in Erie and Pittsburgh. he 
age of the subjects ranged from 17 to 39 years, with a mean 9 
23 years, and all had completed at least six years of education 


TABLE 1 
Description of the Experimental Groups 


Number Number 
Group of of Location Factors 
Subjects — Tests* 


O 1079 19 Erie and Pittsburgh O  SPOAT Р М 
1 21 25 Dallas and St. Louis OV NSP QA Å pM 
2 99 29 Sacramento O NSPQAL IML 
3 141 15 West Virginia О NSPQ T L 
4 138 25 Philadelphia O NSPQ TFM 
5 275 27 Cincinnati, Detroit, OV N S P Q A 
Cerena, Toledo 
an icago 0 
6 98 28 chap — ОУМЗРОАТЕМ 
7 594 25 Composite of Groups O V N S P Q A 
1, 5 and 6 T 
8 204 24 Same as Group 2t O NSPQ A 
indicated, 
* Some of the correlational matrices were actually larger than here indica 
because of the inclusion of age and education as additional variables. ame 


- ^ n S: 
, t This group includes all of Group 2 plus additional subjects from the 
training course for whom data on five tests were not available. 


EA 
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In the remaining eight experimental groups, all of the subjects 
were males and most of them were trainees enrolled in Voca- 
tional Education National Defense Training courses. The age 
of the subjects ranged from 17 to 39 years, with a mean of 28 
years. The mean number of years of education completed was 
11, and ninety-nine per cent of the subjects had completed 
between 8 and 16 years. It is estimated that about five per 
cent of the sample were Negro, and the rest white. 


General Description of the Tests 


A total of 59 different tests were employed in the several 
factor analysis groups. Fifty-four of these, 48 paper-and- 
Pencil tests and six apparatus tests, were constructed by the 

ivision. The other tests were the O'Rourke Survey Test of 
Vocabulary (Form X4), the Revised Minnesota Paper Form 
Board (Likert and Quasha), the Minnesota Spatial Relations 
est, the Minnesota Manual Dexterity Test—Placing, and the 
innesota Manual Dexterity Test—Turning. In addition to 
the tests, age and education were included as variables. 
ws, Bener al, the tests constructed by the Division are speed 
ve oe time limits for the most part in the neighborhood of 
tent 2e The individual tests are homogeneous in con- 
мерак It was planned to combine them into batteries. The 
“mplo t they are intended for use in offices of the United States 
Beg t Service, chiefly for industrial workers, is a central 
Or “бү а There was no need to construct many verbal 
evelo ellectual” tests. Emphasis has been placed instead on 
mn id of tests of perceptual and spatial ability and e 
to hay, y. It was intended to construct tests that appeare 
not cde validity for occupations, but, on the other hand, were 
of the nit pe to specific jobs as to impair the applicability 
at red for widespread use. All the tests are so constructed 
ney can be easily administered by personnel without 


exten 5 
Sve technical training. 


Th The Factors 
Urstone's methods (2, 3) were employed to extract the 


Centr " y 
old factors from the correlational matrices and to rotate 
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them to a meaningful structure. For each group, a solution 
was first obtained which satisfied the criteria of simple struc- 
ture. The effect of maximizing the number of zero loadings on 
as many factors as possible is equivalent to explaining test 
performance by a minimum number of factors. Simple struc- 
ture is essentially the factor analysis analogue of the doctrine 
of parsimony. 

It was discovered in each group that the first solutions had 
very nearly orthogonal structures. In one group, for instance, 
the largest correlation between factors was .099. The factors 
in an orthogonal structure are entirely independent and un- 
correlated; when the factors are correlated among themselves 
the structure is said to be oblique. Since the structures were 
very nearly orthogonal, and inasmuch as the solutions were 
not so exact that different investigators would have obtained 
identical correlations between the factors, it was decided to 
impose an orthogonal structure on each group and the rota- 
tional process was continued until this was achieved. There 
is an important advantage to the final solutions so obtained: 
Comparisons of the results are rendered less ambiguous, !n 
that reference can be made to factors which bear an identical 
relation to all other factors in each group. 

The smallest number of common factors established in апу 
group was seven, and the largest was ten. In all, eleven differ- 
ent common factors were found. Preliminary results from each 
group were applied to all the others, so that in a few groups 
more factors were determined than could be justified in any 
one of these if conducted independently. Consistent results 
were obtained from the several correlational matrices, in that 
the factors common to a related group of tests could always 
be demonstrated regardless of the composition of the remainder 
of the experimental battery. The loadings of a factor on А 
test for different groups varied to about the same extent aS 
correlations for identical pairs of tests in the different groups: 

Among the factors most readily established were the verba 
(V), numerical CN), and spatial (S) factors. Although only 
three verbal tests appeared in the experimental batteries, the 
fact that their loadings ranged as high as .58 and did not fa 
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below .40 serves to identify this factor. One of these tests is 
the O' Rourke Survey Test of Vocabulary. Another is a prov- 
erbs test which measures the ability of the subject to under- 
Stand proverbs. The third is a test of antonyms and synonyms. 
The numerical factor (V) was found to be present in a wide 
Variety of numerical tests: A test of arithmetic fundamentals 
(addition, subtraction, multiplication, and division); a spe- 
Clalized test of decimals and another of fractions; and reasoning 
Problems, in which the subject solves verbal arithmetic prob- 
lems, Two speed tests in one-digit arithmetic, one in addition, 
Subtraction, and multiplication, and the other in addition only, 
аге also classified as numerical tests. 
The spatial factor (S) appears in about a dozen of the 
9ге Paper-and-pencil tests, and was also found in the 
bu Minnesota Paper Form Board and the Minnesota Test 
Patial Relations. The Division tests consist of two- and 
in why umensiona] figures. Examples are surface development 
Constr; а Pattern is matched with a three-dimensional object 
ical узми from it, a picture test of the assembly of us 
o E another on fitting together abstract geometrica 
tern, Ata a test in selecting the mirror image of a line pat 
factor (4. toe remarks on the visual character of his ара 
Similar] т 9-80) and the space tests in these studies ee! е 
ion E. escribed. Kelley’s description (1, 10), шер а- 
Nees in Patial relationships in so far as independent of di = 
Percept ee acuity,” is also applicable. The existence ae 
large ee tends further to limit the spatial factor. e 
Jectiong ат of tests described above, all with ag pro- 
SPace E this factor, make it easily identifiable as the same 
пе a found by other investigators. ret e 
-“'Pretatio ee found in these studies presents cae les 
Is Tesent -— This factor was found in each of the groups D 
Ests Which Significant amount in about two dozen tests. l i 
all of the have significant projections on this factor = ude 
Speeq ee tests, all of the numerical tests except the two 
tests, S of one-digit arithmetic, and almost all of the spatial 
Word me € factor was also present in a letter series test, a 
Mory test, and a perceptual relations test; this is in- 


е 
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teresting because none of these tests have significant projections 
on either V, V, or S. It appears to have some of the properties 
of Spearman’s G, but the two-factor theory has no place for 
group factors like V, N, or S. On the other hand, this factor 
has a wider significance and is more persistent than either 
Thurstone’s А or J (4, 86-88; 5, 158-159). It appears to 
possess many of the properties that teachers, test examiners, 
and clinical psychologists would attribute to “intelligence.” In 
Table 1 this factor has been designated, noncommittally, as 
Factor О. 

A matter of interest, relating to current theories of general 
intelligence, is that this factor has been established in a sample 
of adults, ages 17 to 39. This tends to dispose of some theories 
that this factor could be established only among children, and 
that it amounts to a common maturational factor. The pro- 
jection of age on this factor in this study is consistently nega- 
tive, being about – 250. The loading of У on age, on the other 
hand, is consistently positive, about .320. From this one could 
possibly conclude that older individuals have greater facility 
in expressing themselves regarding familiar situations, and that 
younger individuals have greater facility in coping with new 
situations. 

Among the remaining factors found are two perceptual 
factors, PandQ. Pis present in a test on matching figures ? 
various sizes and shapes, in a test on matching shaded figure? 
in a test on distinguishing figures which differ slightly in length, 
width, area, size of angle, or degree of curvature, and in many 
of the tests which also measure spatial ability. Altogether 1t 
appears to be present in over twenty tests. The factor Ке 
present in about nine tests, including a name comparison test 
and a number comparison test similar to those in the Minnesot? 
Test of Clerical Ability, the arithmetical decimals test, a coding 
test, and a test on number copying. It is also found to some 
extent in several of the other tests which measure verbal °F 
numerical ability. It may be further noted that the test © 
number comparison appears to measure P as well as О a ll 
that the test of name comparison appears to measure О as WC 
as Q. Another comparison test is one in which pairs of geo" 


үч, 


Hy 
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metrical figures are determined to be the same or different. 
Results are not conclusive but indicate that this test measures 
P and not Q. 
_ The essential difference between the two perceptual factors 
I5 not entirely clear. For the most part the tests measuring Р 
Involve geometrical figures while the tests measuring Q involve 
Words and numbers. Another way of interpreting the differ- 
ence is to say that the tests which measure Q deal with ma- 
terials emphasized in formal education, while the tests which 
measure P deal with materials to which little or no school time 
18 devoted, The loading of P on age is about — .400; the loading 
9f Q on age has not been positively established but is much 
Nearer zero, Education has positive projections on both P and 
» but it is much higher on Q. | 
п aiming factor (A) was found in five paper-and-pencil 
adaptations of standard laboratory tests. In one test, the sub- 
J€ct crosses the bars of a series of letter H's without touching 
te sides of the letter. In another test, the subject is required 
to make line strokes in squares. Apparently what is involved 
|j ассцгасу or precision of movement. Whipple (6, 147-151) 
аз applied the term “aiming” to various laboratory tests which 
“sure accuracy of movement. 
n the process of establishing orthogonal structures the 
SPeed factor (Т) was found. It was discovered in one of the 
‘St groups that the factor A could not be established as a 
Aor orthogonal to all the others. Leaving it in an oblique 
Position would signify that the factor A, as then established, 


measured something also measured by the other factors. It 
Due hypothesized that what was being measured in common 


do , QUickness or rate of movement, or speed. This was un- 
i tedly a general factor in a classical sense, since all the tests 
hie, are speed tests. A general factor — та а 
Sstabli n sımple structure, and therefore the ips € P" 
With i ed arbitrarily as a factor orthogonal to 7 t ae ES 
in an © Introduction of this factor, A was easily establis 
orthogonal position without violating simple structure. 
factor factor Т has less consistent loadings than any other 


> ut approximately forty tests have significant projec- 
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tions on it in one or more groups. In general, it is easier to 
establish the existence of a phenomenon when it is sometimes 
present and sometimes absent than when it is always present. 
Related to this difficulty of establishing qualitatively the exist- 
ence of this factor is undoubtedly the lack of consistent quan- 
titative results. Among the tests with the highest loadings ОП 
T, however, are consistently those with moderate loadings on Ai 
From this it follows that an increase in precision in the aiming 
tests results in a decrease in speed. A new speed test was con- 
structed which demands practically no precision, and another 
is under construction demanding a large amount of precision. 
These tests will be included in future factor studies. 

Two dexterity factors resulted from the apparatus tests: 
These have been designated F and M for finger dexterity and 
manual dexterity, respectively. An interesting finding Was 
that the Placing and Turning parts of the Minnesota Test 0 
Manual Dexterity had practically an identical factor compost 
tion, being almost pure tests of manual dexterity. Substan- 
tially the same finding obtains for Parts I and II of the Peg 
Board Apparatus developed by the Division. In Part I pegs 
are moved from one set of holes to another set of holes using 
both hands simultaneously, and in Part II the pegs are tran?" 
ferred from one hand to the other and replaced in the hole 
upside down. Another apparatus test involving fine assembly 
work measures F. However, disassembly of the same parts E 
a composite of F and M. F is significantly present in five tests 
and has moderate loadings on two others; and the existence © 
M is readily established from its presence in significant amount 
in five tests. 

A factor L was tentatively established in two of the factor 
studies, for four or five different tests. One of these is an ana 7 
ogies test consisting of line drawings and another is а verba 
test in following directions. All of the tests with significant 
projections on this factor require the solution of problems by 
formal rational processes. This factor is a narrow reasoning 
factor and is being called the Logic factor. Age has a loading 
of about – .350 on L, and Education has a loading of + -350 on L. 

It had been originally intended to include both a wor 


jal 


+ 


S 5 
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memory and a picture memory test in order to measure a 
Possible memory factor. Unfortunately, only one memory 
test was administered, and no memory factor was found. 


Discussion 


Group factors have great significance for vocational coun- 
seling. This point may be developed briefly. If only a single 
general factor accounts for vocational success, all that one could 
do is establish a hierarchy of jobs, in order of general ability, 
as was done with the Army Alpha results after the last war. 
Tt would not be possible to distinguish, for instance, a potential 
awyer from a potential engineer, assuming that engineers and 
AWyers possessed on the average an equal amount of general 
ability. If we assume, on the other hand, that engineering 
"equires more ability than law, then it would follow that every 
Individual would find law easier than engineering. 

п the other hand, if one were to postulate that every task 
required its own separate specific ability, it would be impossible 
4 the Vocational counselor to assess all these abilities, and to 
Predict all the vocational aptitudes the applicant might possess. 
i s ith group factors perhaps only a few hours of testing are 
5 Nr ned to sample the significant aspects of behavior. а 
ау that each occupation requires а different set o 
tio udes from that of every other occupation, those occupa- 
ns with similar requirements could be grouped together into 
Then, on the basis of a relatively small number of test 
Tes, prediction could be made for a number of occupations. 
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THE INTERESTS OF FOREST SERVICE MEN 


EDWARD K. STRONG, JR. 
Stanford University 


THE top executive who has worked up from office boy has 
a profound belief that others could do likewise if they only 
would try hard enough. Is this true? Is willingness to work 
the major consideration or must abilities and interests also be 
taken into account? Suppose every nonsupervisory position 
Was filled with a well-qualified man, would that insure a suffi- 
cent number for promotion to supervisory positions as vacan- 
Cles occurred? 

Many organizations, both public and private, maintain the 
policy of recruiting top management by promotions from below. 
Other organizations select men specifically for executive and 
Administrative work and promote relatively few upwards from 

elow, Personnel practices relative to selection and training 
reflect the policy of the organization involved. In far too many 
Cases, however, no one in authority has ever definitely decided 
What the policy should be nor are the facts available upon which 
Such a policy should be formulated. 

Some data are given below which suggest that in the forest 
Service the interests of district rangers and administrators differ 
3D appreciably that it is questionable whether many of the 
9rmer will or should be promoted to management. And it is 
urthermore questionable whether there are enough men in the 
т bracket with the interests of administrators to supply the 

I with properly qualified men at the top. - 
es S 1936, through the cooperation of the United States For- 
ii 410 members of that service filled out the Voca- 
Scale nterest Blank. On the basis of these blanks an interest 

° Was developed to measure the degree to which a man has 
meg wrest of forest service men rather than the interests of 

1n general. 
uch blanks may also be scored so as to reveal how similar 
157 
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a man’s interests are to the interests of men in 35 other occu- 
pations. Thus, it is possible to say that a particular ranger has 
more the interests of an engineer than of a forest service man 
and to say that a supervisor has not only the interests of a for- 
est service man but also to a lesser degree the interests of a 
production manager, a personnel manager, a public administra- 
tor, and an office man. Such a supervisor might be expected 
to function somewhat differently from one who had an equally 
high forest service interest but whose secondary interests were 
those of an engineer, farmer, aviator, and policeman. 

The data in this report are based upon the original group 
of 410 men plus about 50 additional cases of supervisors an 
regional and Washington administrators, obtained in 1941 in 
connection with a study of public administrators, financed by 
the Committee on Public Administration of the Social Science 
Research Council. 

The data pertain to interests—what a man likes and also 
dislikes to do. The data do not include measures of general or 
specific abilities. Interests and abilities must both be con- 
sidered before a final answer to the questions before us may be 
obtained. But what a man is interested in does play an impor- 


tant role, and it is this aspect of his behavior with which we are 
concerned here. 


Scores of Forest Service Men on the Forest Service 
Interest Scale 


As stated above, the forest service interest scale is based 
upon the records of 410 men, whose average age in 1936 was 
38.5 years. Approximately half were district rangers; all bus 
three of the remainder were assistants to supervisors, assistant 
and associate supervisors, and supervisors. In terms of гап 
the criterion group averages just above that of district range" 
Forest service interests as measured by the scale represent the 
interests of men from district ranger to supervisor. 

Table 1 gives the forest service interest scores of 430 mer» 
distributed by age and rank in the service. 

1 The classification of forest service men and the follow-up referred 30 ta 
were made by Paul P. Pitchlynn, formerly assistant regional forester at an 


A aa i B ely 
cisco. The 410 blanks constituting the criterion group were also obtained larg 
through his efforts. 
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TABLE 1 


Number of Forest Service Men by Age and Rank; and Mean Forest Service 
Interest Score of Each Sub-Group 


Age 

Total 

5 30 35 40 45 50 55 60 
District N 37 38 40 30 24 D 7  .« 189 
'angers M 532 525 5L5 475 460 479 382 ... 500 
Hence NW éd 4 $ o e Ro ox Шр 
pru M 478 60:7 453 oc. sxx sae SOO... OR 
nt. N 10 28 18 4 9 3 1 MOS. 
Supervisor М 541 527 521 405 462 463 370 ... 508 
Spiess N т 4& 5 ouo 
uberior M 405 517 478 500 m: SEO su a 497 
visor N  .. 18. id? dL 30 11 О ше 
Pee M 475 475 511 483 489 413 ... 475 
N t X 319 2. 3s 20 
P-7 p M 50.0 50.0 442 310 373 453 
3 N P d 4, 5957 16 
Total М Loin n. 440 500 395 306 305 348 
N 59 93 8&2 SO 68 4 5 430 


31 
M 521 518 504 478 473 465 370 346 4861 


. 
F ederal È P-7, and P-8 refer to grades of positions in the professional service of the 
Tesponsibil ment, many of which entail a considerable amount of administrative 
the time p and with respective base salaries of $5600, $6500, and $8000 (as of 
. tOnl E € administration of the test). d 
Criterion Y 3 of the P-6 and none of the P-7 or Р-8 personnel were included in the 
cluded eee: If they were excluded and 13 cases, not so far classified, were in- 
> ае mean standard score of 410 criterion cases would be 50. 


all ne data make clear that such scores decrease with age for 
to only Tes in the service. The decrease in score amounts 
157 from rom age 27.5 to 47.5 years, whereas it amounts to 
Dum 47.5 to 62.5 years. bed i 
1 à phen Se with age in score on the forest service interest iex e 
similar ae strikingly peculiar to the profession. Data 
tions those in Table 1 have been published for 29 occupa- 
Meludin ; e average decrease in score for these 29 sie r 
and S7 ye Crest service, amounts to 1.4 between the ages О 
еге ae whereas the decrease is 15.7 for forest due ran 
Personne] appreciable decreases in score amounting to 10.> tor 
Managers, 7 for aviators, 5 for realtors and physicians 


and 4 


о : : DEOR 
ng i r life insurance salesmen. The only occupation exhibit 


Der Р Eds 
7 16аве of score on its own scale is that of minister, where 


2 E ы 
Versity Pre Strong, Jr. Vocational Interests of Men and Women. Stanford Uni- 
» Table 75. И 
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the increase amounts to 6.9 score points. For most occupations 
age affects interest scores very little, but for some reason the 
reverse is the situation in the forest service group. 

This phenomenon explains in part why a few successful for- 
est service men score low in the forest service interest scale. 
They score low because they are older men and all older men 
average considerably below 50.* 

The question naturally arises as to why forest interest scores 
decline with age among forest service men. Three possible 
explanations may be considered: first, such interests actually 
decline with age; second, there has been a change in the type 
of men entering the service and the younger men, constituting 
the larger proportion of the criterion group, have largely deter- 
mined the norms for the group; and third, men with high scores 
leave the service in later life to a larger degree than men with 
lower scores. 

The preponderance of data in our possession does not sup- 
port the first explanation, that the kind of interests possesse 
by forest service men naturally decline with age. For example; 
when samples of 25-, 35-, and 45-year-old men drawn at random 
from the population are scored on the forest service interest 
scale the mean scores are, respectively, 28, 27, and 31, indicating 
a slight rise in score with age. 

How about the second explanation? In the early years of 
the forest service few men were college graduates, whereas in 
recent years many have been recruited from colleges, particu" 
larly from schools of forestry. Has this change in selection 
affected the interests possessed by forest service men? 7 

The relationship of age to amount of education among 18 
district rangers is as follows: 


For 


3 The standard score of 50 is the average obtained by forest service теп. j C. 
convenience scores are often expressed by the three letter ratings of A, B, an 
An A rating (scores of 45 to 75) means that the individual has the interest 
successfully engaged in the occupation; a C rating (scores below 30) means 16 
not have such interests; and a B rating (30 to 44) means he probably has those к 
ests but we cannot be as sure of the fact as in the case of A ratings. Note that 
rating does not say a man is not interested in a particular occupation; it says he tion 


not find interesting a whole range of activities which successful men in the occUP 
find interesting. 


4 [bid., р. 272. 


ter- 


21 
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Age Number Years of education 
20-29 years 38 16.1 (college graduation) 
30-39 years . 77 13.6 (nearly 2 years of college) 
40-49 years . 54 10.2 (2 years of high school) ' 
50-59 years . ; 18 9.0 (1 year of high school) 
OER КИЛДЕ 187 12.7 ((2/3 year of college) 


On the basis of these data one might surmise that the decrease 
1n forest service interest scores with age is attributable to lesser 
amounts of education. If, however, the age factor is partialled 
out we obtain the following interest score ratios for the six sub- 
Broups in terms of amount of education: 


6 to 8 years, grammar school ............-.... 83 
l to 2 years, high school ... 94 
3 to 4 years, high school ... w 


1 to 2 years, college ...... 
3 to 4 years, college ........ .. 
l to 2 years, graduate work... 105 


There is here no indication of increase in forest service interest 
Score with increased amounts of education, except that those 
ne grammar school education do score lower than the 
rant a ег. But there are too few cases in this category to | 
is no nea upon the exception. Among aperiens Ps 
and es егепсе in forest service interest score between 24 co еде 
of edu noncollege graduates. Apparently increase in а 
Still bares has not affected forest service interest score. tis 
wie that there have been other changes in selection 

‘des education and that these undisclosed factors contribute 


tod à 
of ge е in score with age but we have uncovered no proof 
is. 


see third explanation, that forest service scores decline with 
arger eu older men with high scores leave the poe ae 
only h €gree than those with low scores, can be e is e 
айк а long time follow-up of the men so far teste = 
at tay, Interest Blank. Evidence presented below indicates 
Scores «motions, on the average, go to men with lower oe 
Wiens mechanical pursuits and higher scores seg" m 
e Mer Interests. Such a hypothesis is tenable as a 
interest "onship of forest service and public pore 
at so 5 15 concerned, for they correlate only .21, whic lse 
low i me men with high interest in one of these fields will have 
"terest in the other. It is therefore possible that some 
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men with high forest service interest scores have left the service 
when they failed to receive a promotion and that there have 
been enough such cases to explain the decrease in average scores 
with age. 


Occupational Interests of Forest Service Men 


So far we have considered only how far forest service men 
score on the forest service interest scale. Consider now how 
they score on other occupational interest scales. 

Table 2 gives the percentage of rangers, supervisors, and 
administrators who rate A and B + (scores of 40 to 75) in inter- 
est for 36 occupations. Such percentages indicate how many 
forest service men definitely have the interests of men in those 
occupations. For example, 88 per cent of rangers rate A and 
B+ on forest service interest in comparison with 82 per cent of 
supervisors, 55 per cent of P-6 officials, and 38 per cent of p-7 
and P-8 administrators. As explained above, much of these 
differences is attributable to increasing age as we go from dis- 
trict ranger to P-7 and P-8 administrator, but we suspect 
not all. nidi 

Forest service men have in general the interests of skille 
tradesmen (Group IV), particularly farmers; of production 
managers; of engineers; and of public administrators. Few О 
them have the interests of scientists (Groups I and ID, office 
workers (Group VIII), salesmen (Group IX), lawyers-writers 
(Group X), and men engaged in social service (Group у). 

Commenting upon Table 2, a former administrator of the 
service writes: “It explodes the old fiction that Forest pei 
men are scientists. True, some of us have had a little scient! Й 
training and some perhaps is а good thing, but we attempt (s 
recruit scientists as rangers. Those we get go in one of e 
directions: some quit, some get transferred to research wor } 
and some become frustrated and are no good to themselves ? 
the Service.” f 

There are some striking differences between the interests ? 


At ow the 
district rangers near the bottom of the organization 2% idy 
administrators at the top. Such differences are indicated ‘and 

ti 


well in Table 2, especially in the case of those occupa 
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TABLE 2 


Percentage of Forest Service Men Rated A and B+ on Occupational Interests 
(i.e., Percentage Scoring 40 and Above) 


49 44 20 P-6 a 
: н а Washington Washington 
Group Occupation —— Supervisors & Regional & Regional 
57 
ПІ Production Mer. 64 55 55 
IV — Mechanical Activities 38 
Forest Service 88 z 55 6 
Farmer 86 11 Е 0 
Carpenter 35 27 35 6 
Aviator 36 13 15 0 
Printer 28 23 35 6 
Policeman 34 21 25 6 
, Math. Sci. Teacher 34 2 
I Physical Sciences 40 38 
Engineer 36 A 35 31 
Chemist 24 0 5 6 
Mathematician 0 
УШ Office Activities 16 30 13 
Office Work 26 28 20 12 
ВапКег 18 13 1$ 13 
ccountant 10 29 20 25 
Purchasing Agent 22 
IX вае Activities 29 10 0 
Realtor 22 25 20 0 
Sales Mgr. 6 11 15 6 
Life Insurance 8 8 20 32 
ч President 16 1 
Social Service 85 100 
Public Administrator 38 T 50 38 
Personnel Mgr. 14 4 15 6 
Y. Physical Dir. 14 11 30 19 
ocial Sci. Teacher 8 11 25 19 
City School Supt. 2 7 31 12 
Y.M.C.A. Secy. 4 5 5 6 
Minister 0 
X — Linguistic Activities 14 10 31 
uthor-Journalist 8 25 5 44 
awyer 6 12 5 19 
ү Advertising Мап 4 M 0 0 
1 usician 6 0 
Biological Sciences 15 25 
ysician 16 Ei 10 25 
chitect 8 9 5 0 
entist 10 2 10 6 
Artist 2 0 10 12 
VII В Sychologist 0 
Certified Public 2 0 6 
ccountant 0 


forest 

inter” The 49 rangers are a selection from 190 cases, so selected Har Фе XT нА pu 

the 195 Score and standard deviation of the 49 cases are Pr TOU, EI p LUE 

Ps 50 cases, he 44 supervisors were similarly selected from E eM ALT 

Btoup d 16 P-7 and P-8 administrators are all the cases in our por a. ey 

and Р S tins 5 men stationed at Washington and 15 at ngon i Watling. 
ÉrOup contains 5 regional foresters and 11 men statione 
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interests in which a fair number of forest service men are inter- 
ested. Such differences are, however, better shown by mean 
scores, for such take into account the men who score low as well 
as those who score high. Differences of four ог more in mean 
scores between district rangers and P-7—P-8 administrators 
are given in Tables 3 and 4, the former giving the occupational 
interests which decrease and the latter the occupational inter- 
ests which increase, as one goes from district ranger to adminis- 
trator. (In nearly every case the mean scores of supervisors 
and P-6 administrators lie very close to a plotted line connect- 
ing rangers and P-7—P-8 administrators. This fact adds sup- 
port to the conclusions drawn from the data of district rangers 
and P-7—P-8 administrators alone.) 

The interests which decrease (Table 3) are for the most part 
typical of mechanical activities, whereas the interests which 
increase (Table 4) are much more varied, being associated with 
administrative work, law-journalism, and social work. 

The differences in interest scores in these two tables indicate 
that administrators differ in their interests from district rangers 
and, to a lesser degree, from supervisors. The differences imply 
that administrators are selected on a different basis from that 
used in the original selection of district rangers. This relation- 
ship will be found in most organizations, for administrators 
differ from the rank and file both in abilities and in interests- 

One of the most notable differences in interests between dis- 
trict rangers and P-7—P-8 administrators is in the interests © 
public administrators. Only 38 per cent of district rangers rate 
А and В + compared with 66 per cent of supervisors, 85 per cent 
of P-6 administrators and 100 per cent of P-7—P-8 admins- 
trators. 


SDi sees ЕЕ : s ios of 
5 Differences of 7 and 8 are statistically significant, i.e., have critical ratios 


3.0 and over, judging from a number of calculations; for example: 


Forest Public Personnel 
Service Harmer Administrator Manager 
Dif. CR. Dif. CR. Dif. CR. Dit CR 
Ranger vs. Super. 50 2.3 50 28 45 24 558 2 
2 P 80 29 70 30 15 48 15 27 
“ P-/—P-8 160 61 145 76 120 59 HO 15$ 
Super. vs. P-7—P-8 125 48 95 47 70 34 55 L 


pas. 
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recu TABLE 3 
ж Interests in which Rangers Score at Least 4 More than P-7—P-8 
inistrators; also Differences in Scores of Younger and Older 
Rangers and Supervisors 


Difference in scores 


District P-7—P-8 Difference of 30- and 50- 
year-old men 


rangers* 
Rangers Supervisors 

" 52.5 36.5 -16.0 -10.3 2.0 
47.5 33.0 -145 2.3 2.5 

38.0 25.0 -13.0 - 70 -1.0 

37.5 23.5 -140 7.5 40 

36.0 14.5 -21.5 7.7 9.5 

35,5 24.0 -1L5 st 5.0 

35.0 29.0 - 60 - 5 3.5 

33.5 29.5 - 40 2.5 = 5 

33.5 29.5 - 40 2.7 -10 

31.5 26.5 - 5.0 3 2:5 

ee. 30.0 21.5 - 85 11 2.0 
r.. 275 23.0 - 45 - 32 - 35 


Th k 
Brees Ver. Tank order of occupations based on A & В + ratings of rangers in Table 2 


ome with rank order based on mean scores of rangers, some of which 
is table and in Table 4. (The correlation between the two is .976.) 


пете Public administrator scale is new. It is based on the 
cludeq in Bü 18 men engaged in public administration. In- 
€ forest Se group are 46 supervisors and administrators of 
rvice. The data in Table 2 are based on these 46 


M -——— ———ÀÀ —————ÉET— С D 
— cx a c mes 


T TABLE 4 


Oc 
Upati 
ton, ] à 
/ Ебу Мей, in which Rangers Score at Leas 
trators; also Differences in Scores of Yo 
Rangers and Supervisors 


t 4 Less than Р-7—Р-8 
unger and Older 


Differences in scores 


District of 30- +e 50- 
i =i men 
rangers P-7—P-8 Difference уеаг-0! 
Rangers Supervisors 
39.5 51.5 12.0 -11.0 -40 
30.5 35.5 5.0 7 -50 
28.0 39.0 11.0 -15.8 -40 
27.0 340 70 = 28 -3.0 
25.5 37.5 12.0 = 5,5 d 
> 25.0 33.0 8.0 - 40 —5.5 
21.5 340 12.5 = 213 E 
21.5 25.5 40 223 25 
20.5 25.5 5.0 = 33 ; 
16.0 22.5 6.5 - 48 35 
18.0 30.0 12.0 BET -65 
13.5 25.5 12.0 - 98 -30 
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men and in addition 34 supervisors-administrators and 49 
rangers not included in the public administrator criterion group. 

Scores on the public administrator scale correlate highest 
with scores on the scales of personnel manager (.75), city school 
administrator (.55), Y.M.C.A. secretary (.53), social science 
teacher (.53) and Y.M.C.A. physical director (.52). Since 
forest service administrators rate so very much higher than 
rangers on the public administrator scale it is to be expected 
that they will score higher on the social service occupations. 
The data in Table 2 do not show such increases except in the 
case of personnel manager. This is true because, with the ex- 
ception of personnel manager, few forest service men score high 
on these occupational interest scales. Examination of mean 
scores shows, however, a steady rise in these interests from 
ranger to the P-7—P-8 level (see Table 4). But it must be 
emphasized that at the top level the mean score is 39 on the 
personnel manager scale, 34 on the city school administrator 
scale, 29 on the social science teacher scale, and 25 to 22 on the 
other three social service scales. Forest service men on the 
whole score low on the social service scales, scales which are 
related significantly to the interests of public administrators: 
This is true even of the forest service administrators. 


Younger Men More Similar to P-7—P-8 Adminis- 
trators than Older Men А 

The fourth column in Tables 3 and 4 gives the difference 12 
mean scores of 30(25-34)- and 50(45-54)-year-old ranger? 
Most of the differences are reversed from those between dis- 
trict rangers and P-7—P-8 administrators. That is, the older 
rangers differ from top administrators more than younger 
rangers. The same conclusion applies equally well to younger 
and older supervisors (last column in Tables 3 and 4)- 

The following correlations between interest profi 
groups of forest service men tell the same story. 


les of 


30- vs. 50-year-old rangers ..................:: .861 
30- vs. 50-year-old supervisors ............- .888 
30-year-old rangers vs. P-7—P-8 ...........-.-- 402 
50-year-old rangers vs. P-7—P-8 ............... .105 
30-year-old supervisors vs. P-7—P-8 ..........- .769 
50-year-old supervisors vs. P-7—P-8 ..........- .569 


P-6 vs, .P-—P-8. cioe secius tee ves solea pe 578 


INTERESTS OF FOREST SERVICE MEN 167 


The interests of younger and older rangers are quite similar; 
the same is true of supervisors. Rangers’ interests are not very 
Similar to the interests of P-7—P-8 administrators, but the 
Interests of the younger rangers are somewhat more similar than 
those of the older rangers. Supervisors’ interests are much 
more akin to the interests of top administrators. Particularly 
18 this true of the younger supervisors, whose interests are more 
Similar to those of P-7—-P-8 men than are the interests of P-6 
administrators » 
tra The differences in interests of district rangers and adminis- 
tors Suggest that the man who is most typically a ranger 1s 
UR likely to rise above the rank of supervisor and that promo- 
'ons above the rank of supervisor are in terms of interests which 
for, possessed by only a minority of district. rangers. One 
m. E administrator comments, “Time after time I үө seen 
тей. rangers promoted only to lose interest and poi 
em cre, or at least no longer outstanding. | А уегу ое! 
Tes ere as elsewhere is “how to determine in advance who wi 
Pond to promotion and who will not.” | 
Perma Rene know about interests indicates that they are к 
they fee especially among adults. There are W^ w. o 
tions iod changed appreciably but such appear to а 
istrict neers and a employ the data on sous ds. 
attribut ra and supervisors as indicative of the ch ET 
Tong Fs le to increasing age, then such changes are i EN 
Te you 'rection—older men are less like xin Aon 
ent mw men. [f we assume that interests are fairly p ; 
i that the above changes are not attributable to increas 
Older then it would appear, as suggested above, that iue 
St en with distinctly ranger but not administrative in 


i . . t 
E dropping out of the service when promotions are no 


i i t age groups of 

S псе of .297 between the two correlations of differen Е 

Tees о Po has a critical fant 1.4, and the difference of 200 PO Es Y 
Ц signif 1507 has а critical ratio of 2.3. Neither of these 

T ificant, 


ч ut е 
Sider as Posse SUggest that district rangers are well selected for uae m. T ie they 
Dot үр i € interests characteristic of administrators. dud phis 


E the | 
rough pyoponsibiliti ded by in 
Present t nsibilities. They should be rewarded by 
oat wen отоор into a different type of work, but by kee 
9r which they are suited and which they enjoy- 


ping them on their 
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One cannot help wondering if the forest service is recruiting 
at the bottom enough men typical of top administrators to pro- 
vide a good assortment from which in later years to select the 
leaders of the organization. 


Recreational Interests 


In early days the forest service was concerned with the man- 
agement of forest land involving lumbering, grazing, the con- 
struction of roads and buildings, and the fighting of fires. Then 
the public discovered the forests were a wonderful place for a 
vacation. This has focused greater emphasis upon the activity 
of handling people. Under the circumstances it 1s natural ito 
ask, do the men selected for the original purposes of the service 
also possess the interests of men dealing with people? Is there 
any evidence that the younger men who have been selected in 
recent years have more social interests than the older men? 

If we postulate that the occupations in Group V (see Table 
2) typify the men who “handle others for their presumed 2000 
then we can measure the extent to which forest service men 
possess social interests by noting their scores on the occupations 
in this group. Reference to Table 2 makes clear that few for- 
est service men have such interests. The percentages are low 
for five of the six occupations and not at all high in the case 9 
the sixth, i.e., personnel manager. 


TABLE 5 


Interests of Recreational Administrators; also Differences in Scores of 
Recreation Men and Forest Service Men 


Differences in score between 

: " Mean score recreation men an 
Occupational interests of recreation p-8 
admini: istri 7—17 

nistrator District Superv: Вб È 7- 


тапрегѕ 
Public Administrator .... 48 8 -4 3 _ 1 
Personnel; :usesss ae evase +} -16 -10 - 3 14 
Social Science Teacher ... 44 -17 -16 -12 L8 
City School Supt. ...... 42 -20 -15 -12 45 
NLCA: DEY: aris svidi 41 -19 -16 -12 17 
Ү. Physical Director .... 40 -12 -12 -9 2 
Lawyer repasti faciei 36 -10 -1 -5 26 
Math.-Science Teacher .. 35 0 - 4 0 -12 
ТЇ ШШЕ ӨР rossa iai diecast 35 -19 -17 -12 902 


ye 
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If we postulate that the interests of public administrators 

engaged in recreational work typify the social interests which 
forest service men should now possess, we may then employ the 
Interests of recreational men as a standard against which to 
check forest service interests. The nine occupational interests 
Оп which recreation administrators score 35 and higher are 
8iven in Table 5. It will be seen that six of the nine constitute 
the occupations in Group V referred to above. It will also be 
Tecalled that these interests correlate significantly with the 
Interests of public administrators in general. 
. Forest service men score low on most of the interests listed 
1n Table 5 (differences of 8 are statistically significant in almost 
every case). There is better agreement between the interests 
of Tecreation administrators and forest service men as we go 
from ranger to P-7—P-8 administrators, with the exception 
that Р-6 administrators are slightly more similar to recreation 
administrators than are P-7—P-8 administrators. 

When all 36 occupational interests are taken into account 
instead of only nine, as in Table 5, we have the following corre- 
lations between the interest profiles of recreation administrators 
and sub-groups of forest service men: 


Recreation уз. 30-year-old district rangers +++... 0 
" *  S-ycar-old district rangers ... = 
“ 30-year-old supervisors ..... E 
‘ * 50-year-old supervisors * 
и Йй EE, nen я 4 
E MI s pem » 
H Ra NC £A wmv vev E 
та ee there is no relationship between the interests of 
1 


Оп administrators and the interests of rangers, m - 

ming tionship in the case of supervisors and P-7— : 

inis, ^ 1а00, and some relationship in the case of P-6 ad- 
'Stratorg, 


е . B 
jr above correlations between the interests О 


Slight 
adn: 


f recreation 


dm H 1 
in 
the ОП eaters and forest service men may be compared кем 
ei e correlations between recreation administrator: 
Sht other groups of public administrators: 

Recreation vs. Personnel men in public service «+--+ a 

“ “ Social insurance administrators « эм 

“ “ Welfare administrators .-----+-- ue 

« e Publicity men = 5 

« Statistician ....- I " aD 

« 5 Public health officials . aee ei 

« Engineers .........:- UO 


Chemists-Physicists .... 7 
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Seemingly, if the forest service is to handle the problem of 
recreation within the forests it must have men in the organi- 
zation who understand such problems and genuinely enjoy 
dealing with them. Such interests are different from the inter- 
ests of the typical forest man. Two ways of meeting the situ- 
ation occur to us. First, men who possess both types of inter- 
est might be brought into the service. Second, men who are 
possessed of the recreational type of interest might be brought 
into the service to be specialists in this field. The former 15 а 
doubtful procedure because there are not many men who pos- 
sess both sets of interests. (Forest service interests correlate 
with the interests of the occupations listed in Table 5 as fol- 
lows: Public administrator, .21; Personnel manager, ^ 0l; 
social science teacher, —.13; city school superintendent, - 25; 
Y.M.C.A. secretary, —.07; Y.M.C.A. physical director, 39; 
Lawyer, – :61; Math.-Science teacher .68; and Minister, 00 

The second procedure would force a rearrangement by which 
recreational activities would at least be directed, if not сагпе 
on, by specialists. This would not be so convenient as Е E 
present procedure of having rangers carry on all types of activi" 
ties. Whatever the organization, it certainly appears that there 
are not enough forest servicé men with the interests of recre- 
ation men to carry on such work enthusiastically. 

Since the interests of administrators directing public тесе 
ational work correlate significantly with the interests of at- 
ministrators in general, it is possible that adding such men ie 
the forest service might result in increasing the number 9 
younger men who would be selected later on for administratiVe 
work in the forest service. 


Conclusion 
rvice pe^ 
d before 


btainec- 


This report is restricted to the interests of forest se 
sonnel. Abilities as well as interests must be considere 
complete answers to the questions raised here can be О 
The data suggest: 

1. Forest service men have in general the interests O^ 77" 
tradesmen, particularly farmers; of production managers, Ў, 
engineers; and of public administrators. Few of them have 


f skilled 
[o) 
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Interests of scientists (Groups I and II), office workers (Group 
ҮШ), salesmen (Group IX), lawyers-writers (Group X), and 
men engaged in social service (Group V). 

2. The older the man in the forest service, regardless of 
Whether he is a district ranger or top administrator, the less he 
T3 the interests of district rangers-supervisors. Such decrease 
M interest score is apparently not attributable to increasing age, 
pr Shift in type of man entering the service, but to men with 
igh Scores leaving the service in greater numbers than men 
ү» бу Scores. Another bit of evidence in support of this 

P anation is that younger district rangers have interests some- 
лаг More similar to those of top administrators than do older 
us rangers, the same thing being also true of supervisors. 
istrict rangers differ in their interests from administra- 
he forest service. The former have stronger mechani- 
ests and weaker interests associated with law-journal- 
сосы work, and administrative work typified фе 
Dess co Perintendent, personnel managers, president of a busi- 
‘cern, and public administrator. 
Men in Cannot help but wonder if there are enough ee 
higher : € Service with interests of administrators so that the 
Positions can be well filled twenty to thirty years hence. 
inter, UPatively few of the forest service personnel have ps 
“ats the LN among recreation administrators. But in i a 
Tecreat: rests have come to be used by millions of people 
lona] Purposes. | 
ШЧ not some forest service personnel be specifies 
stead ° to handle the recreational facilities of the service in 
already , ^ Pecting this function to be taken care of by ч 
Mterestg eavily loaded with other work and not possessing the 
sale Ot recreational enthusiasts? € 
апа likely that the rank and file of «паеха а ы 
h Should 19 differ in their interests from the execu i РА 
Spe t men be selected to fill the lower positions wi 
Vacancies Some will eventually fill the few higher positions as 
futur "CS occur, or should: some men be definitely selected for 
е > 


cement in higher positions? 
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A GROUP TESTING PROGRAM FOR THE MODERN 
SCHOOL 


WARREN G. FINDLEY 
New York State Education Department 
Pi 15 the fundamental thesis of this paper that the modern 
in is has certain definite functions and characteristics which, 
а Sata light of the findings of educational psychology, call for 
u 1 . . 
istics, P testing program having equally definite character- 


Characteristics of the Modern School 


2 modern school is characterized, first of all, by MX 
е peo iX responsibility for educating all the children of al 
age m. rom school entrance until completion of grade e 
or remain : At present some pupils do not complete grade 1 
tions Б i school until 18 years old, but increasing propor- 
Tésearc PN alien planning is based on this premise, and 
Schools feil eing pointed in the direction of finding why the 
This tiend to hold those that now leave without graduating. 
increas: toward the twelve-year common school has resulted 
© up ng individual differences in the pupil population in 
also Per grades with respect to interests and life goals, if not 
respect to intellectual ability. 
рес ы d important characteristic of the modern school, 
regular уш the elementary grades, is the tendency to promote 
logical 1 Practically all pupils. This is based on the psycho- 
to nding that the individual differences most significant 
c evelopment are social factors associated more closely 
The ас 101081681 age than with intellectual achievement. 
2 the i of regular promotions finds further justification 
Sect “dily observed and experimentally tested deleterious 
This s Tetardation on the morale of the retarded pupil. 
promotions policy has produced increased indi- 
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vidual differences in intellectual development in each school 
grade, so that teachers have been encouraged to “take each 
child where he is” and attempt to further his development as 
much as possible by adaptation of instruction to individual 
pupil needs. 

A third characteristic of the modern school that bears on 
our problem has grown rapidly under the pressure of war find- 
ings that many high-school graduates have failed to maintain 
skill and ability at levels attained earlier. This condition has 
been found especially true of arithmetic. As a result the secon- 
dary school is now taking responsibility for maintaining ап 
developing skills and abilities in basic areas like reading, arith- 
metic, and geography, previously accepted as mastered in the 
elementary school. | 

A fourth significant characteristic of the modern school 1s 
the emphasis on following and guiding total personality de- 
velopment of individual pupils cumulatively and construc” 
tively, and in line with this bringing to parents a clearer and 
less formal report of their children’s progress and mastery © 
the work of the school. The old report card is giving Way to 
interpretative communications, often including other matters 
besides scholastic achievement, calculated to provide the basıs 
for better understanding and cooperation between the school 
and the home. 

Other characteristics of the modern school might be note 
many of them significant—but the above will suffice for 
problem under discussion. 


d— 
the 


Implications for the Testing Program 


The twelve-year common school—more than twelve years 
insofar as kindergarten and nursery school are added at the 
beginning—and the corollary acceptance by the secondary 
school of responsibility for maintaining and extending develop” 
ment of basic skills have made it possible to relax the attit P 
of frequent and grim testing in the early grades. There 15 hs 
longer the necessity to view each year's work as possibly t | 
last. With extended universal schooling it becomes possible 
to plan extended programs of instruction looking toward ? 
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pal pointing up at the end in more comprehensive examina- 
tions of the outcomes of the whole program. The best achieve- 
ment at the end of the twelve years becomes the criterion and 
the justification for practices that lead to such success. In 
the Pupil's whole school career the effort of the school properly 
Ms studying, guiding, and furthering his development to- 
Ward ultimate achievement. | 
Program of testing progress along the main courses of 
E Curriculum is called for if progress is to be guided. Such 
а Program in New York State is called just that, a program 
Te Potress testing" and “progress tests.” Reading Progress 
ests are designed to reflect progress in reading comprehension 
jus, rade 4 through grade 12. Mathematics Progress Tests, 
E Issued, are similarly to reflect progress in mastering gen- 
аа mathematics from grade 4 to as high a grade as may 
iae general mathematical instruction. Similar tests are 
ORE for the understandings, skills and appreciations E: 
skills English, science, health and safety, social ee Ps 
appre “Ppreciation of literature, art appreciation, 
Preciation, 
further feature of this testing program derives from the 
"Cy toward uniform promotions and the emphasis а 
Preting individual achievement both within the school an 


tende 
Inter. 


to | à 
E Parents. In such a program, testing must kw e 
Оп speci i i ble lines rather than 
n mprovable li 
Benera] ,2 P fic and directly imp 


Te terms of subjects. The Reading Progress. T 

епо 2 баев in three identifiable aspects of reading s d 

disc оп: ability to obtain detailed understanding, ability. ; 

ai, Central thoughts i es, and ability to recogniz 

eani oughts in passages, ee 
185 of words. The Mathematics Progress Tests m 


t Wantit . pects of problem solving: social information 
а515 for understanding 
o ncepts that are the b 


tion Atext of mathematical situations or pe САА 
Quireq 1 ability to choose the operation or оре a 
Solve particular problems, ability to bg 
- 7 Опе has sufficient data to solve problems. is в : dui 
throy y levant to the solutions, and finally abi rd i 
abilities the complete act of problem solving, using 


e tests. 
Separately measured in the other parts of th 
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Many standardized tests of commercial publishers covering 
reading comprehension and English usage involve similar sub- 
parts devoted to particular skills in those areas. The Гога 
Every-Pupil Tests of Basic Skills, especially Test B: Work- 
Study Skills, are so organized. . Test В measures progress along 
five lines: reading maps, knowing sources of information, using 
the dictionary, using an index, and reading charts, graphs and 
tables. 

Tests thus subdivided provide the teacher with information 
about the relative strengths and weaknesses of the class as 4 
whole and of individual pupils that permits planning of in- 
struction to meet demonstrated needs. Summaries of achieve- 
ment in this detail help administrators obtain more real under- 
standing of the progress and mastery being accomplished under 
their general supervision. Profile charts showing the achieve- 
ment of individuals and classes in the various skills bring 
out strengths and weaknesses especially clearly to both teacher 
and administrator. They also motivate pupil effort towar 
progress and provide a basis for intelligent understanding by 
parents of their children's status and progress. Under proper 
conditions, pupils may acquire satisfaction and hence motiva- 
tion in plotting their own profiles from year to year in the areas 
tested. 

A long step forward can be taken by establishing an ann 
fall testing program of progress tests. If promotions policy 18 
to result in wide individual differences in every grade and pro- 
motion into a grade is no longer to be based merely on meeting 
minimum standards of achievement, fall testing in basic skills 
and areas of understanding will provide data essential to t e 
teacher as she begins work with her new class. Testing at this 


ual 


1 Тһе standardized testing approach described in preceding paragraphs applies 
well to testing progress in the mastery of skills. It has its limitations when арр ese 
to the content aspects of the social studies and science. Tests of content 17 by 
areas need to be adapted to varying local situations and programs and nee ch 
have a timeliness that standardized materials cannot maintain. It is true that des ar 
of the content of these areas that is important remains the same from year to Y 
and from decade to decade. Standardized tests of these aspects, however, ne 
sarily place a premium on permanent content at the expense of the timely, “he 
the local. Annually prepared school-wide, city-wide, or district-wide tests o 
aspects neglected in standardized tests are essential if standardized tests 
are to be kept from exerting a restrictive influence on instruction through pr 
a biased evaluation of content outcomes. 
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time gives up-to-the-minute evidence on what learning has sur- 
vived the summer vacation to become the basis for further 
Progress. For this very reason, such evidence is also more 
relevant to the administrator’s interest in appraising the status 
and progress of his charges, than that which has commonly 
been obtained at the end of the school year. 

Fall testing avoids the undesirable practice of cramming. 
Teachers will be less inclined to teach for specific tests. Test- 
Ing at this time places a wholesome emphasis on collaboration 
between teacher and pupil to achieve goals of instruction in 
the year ahead. 

If testing in the basic areas is done each year at the opening 
of school in the fall and the results are entered on cumulative 
Tecords kept for each pupil, each teacher is provided not only 
With data concerning the current year's testing, but also with 
evidence of the previous progress of her new pupils. Achieve- 
ment on this year’s test takes on added significance when 
Judged in the light of previous achievement and relevant nota- 
tions about health, social, and emotional development, etc., 
that may be entered on such records. A pupil’s seemingly 
Mediocre achievement this year may represent significant im- 
Provement over his very poor achievement of earlier years, as a 
Tesult of serious effort on his part and, perhaps, special assis- 
tance by his teacher. Such progress, which could not be in- 
ferred from the results of this year’s testing alone, merits at- 
tention and encouragement. 

What has been said above applies generally to the group 
testing of pupils from grade 4 through grade 12. In the 
Primary grades, informal testing is generally to be preferred to 
Standardized testing. In these grades children are still acquir- 
exe lementary skill in reading and the pic: dci 
rt to dealing with standardized test npe s. Printe 
tin, c"! dized tests call for reading to such an extent as many 


m : 4 
eae make reading ability the main factor actually tested, 


t 
Гера A а 
i tim ОЁ the title and content of the кийе кош, he 
li re devoted to testing only the mechanics of subjects, 
d lii Until chil- 


o EE ts : 
dren Mputing in arithmetic or spelling in English. | 
ave matured sufficiently to handle well-rounded tests, 1t 
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is better to omit formal group testing of the mechanical aspects 
of subjects. Until it is practicable in actual test situations to 
relate computation to problem-solving and spelling to effective 
writing, it is better not to risk drawing undue attention and 
effort to those aspects of the subjects that are easiest to test. 

There are, however, some standardized tests that are useful 
in these grades. In the first grade or at the end of kinder- 
garten, “readiness” tests may be helpful to teachers in judging 
pupils’ readiness to undertake beginning study of the printed 
word and other work of the first grade. At the beginning of 
second and third grades carefully selected standardized reading 
tests will aid many teachers to appraise the general level of 
reading ability of each of their pupils, as in later grades the 
more detailed program of progress testing provides data ОП 
many counts. 

A word regarding intelligence testing. Group intelligence 
testing—and group testing is most common—will add little to 
what is learned from a reading readiness test at primary levels- 
The unreliability of intelligence measures from group testing at 
. these levels makes it wise to avoid such testing and the tempta- 
tion the testing brings to enter a very doubtful І.О. on the 
pupil's early record. In the intermediate grades the situation 
is quite different. Pupils are generally adaptable to grouP 
testing and results of intelligence testing provide a helpful clue 
in determining the approach to be made to a pupil having 
difficulty in his studies. One with a high І.О. may be presumed 
to have prospects of considerable improvement if the specific 
source of difficulty in learning can be found; one with a low LO. 
may be expected to have considerable difficulty at least in the 
immediate future in most of his learning and should therefore 
be encouraged in his studies, but not exhorted to seek to atta!” 
tremendous progress. Annual administration of a group in- 
telligence test at the beginning of grades 4, 5 and 6 should yield 
data immediately useful. The three separate testings will re- 
sult in establishing a reliable measure for future as well as im- 
mediate reference. At the junior and senior high-school levels 
it may again be questioned whether anything useful in gener? 
school practice is accomplished by intelligence testing. What 


== 


A GROUP TESTING PROGRAM 179 


has been learned of I.Q. in the intermediate grades and what 
may: be ascertained by fall testing of skills in the upper grades 
os all the necessary evidence of status and progress and 
Sufficient basis for prognosis of general or special success in 
advanced grades. 
о ÉD way of summary, it may be said that the characteristics 
Sch Program of group testing recommended for the modern 
00] are: 
hls twelve-grade testing program corresponding to the 
end "UMS instructional program, with achievement by the 
tota] grade 12 the ultimate measure of effectiveness of the 
Program. 
school Continuous testing of progress throughout the pupil’s 
guidi Career, recorded on cumulative records, as the basis for 
Ing his development toward ultimate achievement. 
SPecific Se of tests yielding significant part scores ine * 
to irs Improvable skills, so that instruction may be relate 
Onstrated needs of individuals and whole classes. 
Eins oo fall testing in each grade so that aad 
€velo ata may be available to all involved in guiding pupi 
t fie teachers, administrators, parents, and ө wr s 
Ookin €s—at a time that is helpful and especially related to 
8 ahead. 
Sparing use of formal tests below grade 4. 
Mediate Se of group intelligence tests annually 
da (grades 4 through 6). ——. 
Sort, ng has been said of diagnostic testing 
Possibly resting is best conducted on an AE 
Togram М owing up leads suggested by the sr ч. 
rou, ЧЕ not as a part of the group testing prog З d 
A Bron P testing of interests and attitudes is not treate : 
Fall P testing program in these areas is well justified, but 
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REPLIES OF PSYCHOLOGISTS TO SEVERAL QUES- 
TIONS ON THE PRACTICAL VALUE OF 
INTELLIGENCE TESTS 


ARTHUR KORNHAUSER 
Bureau of Applied Social Research, Columbia University 


E ae report summarized the answers of a panel of 
е present Specialists to one part of a questionnaire on eve 
Were asked Paper deals with the less technical questions е h 
Teplies of te the same time." Both reports are based on the 
to 85 perso psychologists. The questions were originally = 
he Most ns in this field who were chosen as representative 0 
Selection ые “ехрегїз” оп mental measurement. The 
Specialists wh based upon the pooled judgments of 6 а. 
е Diychols © were consulted for the purpose. The names 0 
ction w; ogists who participated in the poll are listed in соп- 
With the report in the preceding number of this journal. 


me 


Th your ; Question 1 
tica nee pigment, how well do intelligence tests meet the prac- 
the Army, for classifying people as to general mental ability in 


= SE 
my, in schools, and in industry? i 
" In business 
tremely In the Army In schools спа industry 
аай ain eb with a very 
ither wel, unt of error., 7% 19% E 
Nop done. y; Uh. better than | 
b Y wel Out tests... 81 78 2 
Notter th: ell, but somewhat 
bat a Eur tests 12 3 = 
ter ell; lite 26 
N than wit o in 0 0, 5 
Bs a 5 100% 
No ayo ase T т к 
Ax te uus — ‚ i 
2 1015 jou, 
Бору. the 2907141, Spring issue, 1945, рр. 3-15 ivi 
"nore, Su ‘sig to Ties еа эуе obtained for he ies his ‘popularized 
Э r d Jt the pu 1C. 
“шу Publish expert views on the matter to July 1945, as part of a new 
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An interesting result appears when the experts are divided 
according to whether or not they state that they have done test 
work in the armed forces, in schools, and in industry. The 
tests are rated a little higher in each field of application by 
those experts who have not worked in that particular field. 
While the relationships are slight and the numbers small, a 
consistent trend is present. The figures are as follows: 


Has the expert done test 
work in the armed forces 


Has Has not 
The tests do “extremely well" in the Army ............ 2% 13% 
The tests do “not very well” іп the Army ............. 14% 10% 


Has the expert done, 
test work in schools? 


Has Has not 
The tests do “extremely well” in schools ............- 19% 25% 
The tests do “not very well” in schools ..............- 3% 0% 
(nz 59) (п=16) 


Has the expert done 
test work in business 


Has Has not 
The tests do “extremely well” in business ............. 0% 9% 
The tests do “not very well” in business .............. 38% 32% 


Similar comparisons by age of the respondents show no 
tendency for the younger and older to differ in their rating? 
of test accomplishment. The same is true of clinical compare 
with non-clinical psychologists (classified according to their 
report of their own principal types of work). 

When those who indicate *psychometrics" as a field of work 
are compared with others, a slight tendency is observed for the 
psychometricians to use the high and low rating categorie? / 
little more than do the others. Thus, the percentages of psy 
chometricians saying “extremely well” with respect to the value 
of tests in the Army, in schools and in industry respectively are 
9, 27, and 9; for non-psychometricians the corresponding Байге 
аге 3, 9, апа 3. (The perfect 3 to 1 pattern is fortuitous!) 
For the low ratings of test accomplishment, the 2 sets of per- 
centages are these: Psychometricians 14, 2, and 37; others 9, , 
and 27. This tendency may imply that the psychometrician® 
had greater confidence in their own evaluation of test acco™- 
plishment than did the other psychologists. 


PEN 
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Illustrative Comments of the Respondents on Question 1 
Test administration in schools is usually slightly more careful 
than in the Army and in business and industry and conse- 
quently the results are slightly more reliable. 

Better in Army and in business and industry because of greater 
eterogeneity of populations dealt with—less well in schools 
because of greater homogeneity. On the other hand, “gen- 
eral mental ability" is of more importance in schools and if 
adequate testing is done the job can be done extremely well in 
Schools. 
I would place the usefulness of a general classification test to 
Schools and the Army above that to industry. This is because 
the problems in the last field are very frequently highly spe- 
cific. Despite the greater complexity of Army problems I 


Consider usefulness to the Army almost equal to that in the 
Schools on the ground that tests classify the men much more 
accurately than other techniques under conditions where little 


Ume is available for processing, 


more are still too general. l'or example in my present 
ica] где ai intelligence tests meet the 
ы fete ы кчы Асы реге than they do 


for needs for classifying accountants 

I imagi and sub-foremen in the factory. 
rou Пе, in the case of the Army. 
ie dt below age 9 or grade 4, even when care 

higher js » frequently yield unreliable determinations. 
e evels such tests do rather well in school. 

tion ав Measure а type of general mental ability. ые quer 

V. Practica] sf important this rather abstract type cpa Уу a 

in the Ara as, In school the importance 1 fairly с : рш 
) More 11У or in business and industry the importane 


Studied «S tionable, and depends upon the specific job being 


Same would be true, 


fully ad- 
t 


he А | 
Speci retical need is to classify people in te 


ability. 


rms of several 


types of mental ability; not in terms of general mental 


argely dis- 


Y со ED 
carded Сере of general mental ability has been latge no 


пр Work, Mental test experts with experience 1n pra 
(Fi Outside of the school situation. 
two ar there is one very long and crit 
Poses 3Éraphs are quoted, as follows: 
| there is some trait which on 
mental ability,” and Question 2 


ical reply from which 
) Question I presup- 
e may properly ca 


Nally, 


likewise presupposes 


a IS a trait which one may properly a n 
oth traits being (a) unequivocally delne ae d 
9! being detected and measured independently Ol, 


> 
а here such traits! 
ac . Are the c } 

What ee Es test. Rue S de erit m " 
e ions determine ? E dt Ша 


the appropriate operations, then 


have meaning. Only, the answer is to be found in an actuarial 
or contingency-table. It needs no census of experts beliefs 
about what the proper answer would be. If there are no such 
traits as “general mental ability,” or plain “mental ability, 

determinable independently of the tests which you have in 
mind, then every answer that you provide for will be factually 
false, since it presupposes a false assumption. . . . ' 

If the tests are to be used, not to classify people as to “gen- 
eral mental ability” but rather to predict success in training, 
or the like, the answer is to be derived from the coefficient of 
validity, not by pooling personal impressions. Speaking by 
and large, these coefficients have run between (say) .3 and 
.5. For group prediction and rough screening they are valu- 
able; for individual prediction almost worthless.* 
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Í 
Question 2 

(a) How dependably do intelligence tests (the usual group 

tests for adults) measure the mental ability of the individua 

person: How safely can a person accept his test rating as 1 

correct indication of where he will continue to stand in menta 

ability relative to others? R 

(b) What is your answer to the same question, concerning 

the mental test rating of the individual school child? 
| 
' 


Ques. (а) Ques. ®) 

Dependable measure of individual’s ability; сап 7% 

be relied Прот a к» аана нанне cs ouaa Je v 875 76 i 
Moderately dependable; seldom far wrong ...... 74 
Doubtfully dependable; often in error; not to be 17 

accepted without confirmation ............... 18 
Not at all dependable; likely to be misleading; 0 

should not be taken seriously ............... 0 LA 

100% 100% р 

NUTR Gt СО з ы едка чы чийе аний ge ets 74 7) |; | 
No answer ог not classifiable . г 5 


3 t 
1estions abor 
o these 145 
e princip? 


* А comment or two may be appropriate in defense of our qu 
"intelligence tests" and measuring “general mental ability," in reply t 
quotations (4 others expressed similar but less definite concern). Th d others; 
answer is that these concepts are widely used, both by psychologists an eneral 
and that an immense amount of mental testing is aimed at measuring this As 
intellectual ability. Perhaps it would be better, as one respondent виБЁ е, етапі 
employ the term “average mental ability” but prevalent usage seemed to uestions 
the assumption that test experts would recognize what is referred to !n 9 asis of 
about “intelligence tests" and the practical classification of people on the PA ру 
these measurements. There can be no doubt that practical efforts аг $ 
directed at predicting general alertness, adaptability, potentialities for learn: 
The private convictions of particular psychologists regarding the futility hea 
to measure these qualities is scarcely reason enough to justify discarding the 
and refusing to ask questions which pertain to it. + to the 

The other general criticism which is contained in the final quotation, stating 
effect that questions of the type we have asked can be answered simply bY enable- 
the size of correlation coefficients and standard errors, likewise seems ЧЫ: 
There are many confusing statistical results on these matters which make setical 
sary for the "experts" to judge what are the typical or representative $ reri 
findings. Moreover, even after particular figures are accepted, a problem Те pre 
as to the justifiable interpretation of the figures for practical purposes. ative 
cisely these final conclusions based upon the coefficients (plus the less 
evidence) which we were trying to ascertain. 
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Replies to these questions were compared for clinical and 
non-clinical psychologists and for psychometricians versus non- 
PSychometricians. The clinical and the non-psychometric 
respondents are decidedly more harsh in their judgments re- 
garding the dependability of the tests. On Question 2a the 

doubtfully dependable” category (the lowest used by any 
Tater) contained 31% of the clinical psychologists’ answers 
and 39% of the non-psychometricians’ as against 10% and 5% 
or the contrasting groups. Question 2b showed similar but less 
marked differences: 21% and 26% compared with 12% and 
2%. On both questions the top rating category reveals slight 
ifferences of the same kind—that is, the psychometricians and 
on-clinicians tend to be a little more favorable in their 
estimates, 

It seems probable, judging from some of the comments by 
the respondents, that the explanation for these differences may 
le in the fact that a number of the measurement psychologists 
and non-clinicians answered in terms of simple short-run re-test 
reliability coefficients. The others tended to go beyond such 

gures and to consider also the varied and unequal conditions 
affecting individuals and their differing rates of growth over 
Prolonged periods of time. The questions were intended to 
Cover this larger problem rather than to refer to the narrower 
Westion of test reliability in a technical sense. 


Mlustrative Comments of Respondents on Questions 2a and 2b 


st group tests place the non-reader or slow reader at so 
Breat à disadvantage as to make results questionable until 
reading ability is determined. 

ЧЕ test results should be checked by reference to other sorts 
9! evidence such as accomplishment in school and in occupa- 
pon, and supplemented, when there are discrepancies or doubt, 

y tests administered individually. 

pen: of course, is necessary in the presen 

e „Ог low ratings; verification with a more re 
t 18 desirable in such instances. 

nou tests are satisfactory for scholastic needs in school, but 

es = all other school needs. Likewise factors extraneous to 

E ata may invalidate their (test data) classification value, 

Wii low effort may negate high intelligence and vice versa. 

ide diversity in values of different tests. 


ce of unusually 
fined individual 
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It would be better never to give anybody a single score alle 
an “intelligence” score. Various aptitude scores shou e 
used instead. 


In my opinion incalculable damage has been done by the 
dogma of the “constancy of the 1.0” For example, in uin 
school systems a child whose I.Q. at 9 years is 90 is not pon 

to take algebra or a foreign language. Considering that the 
S.E. of prediction is about 9 points, one can see that some 
children are called dull at 9 who would be called normal or 
better at 11; while a very small proportion rated dull at one 
age would be rated near geniuses at another. 


The use of group test results as predictors for individual cases 
is dangerous. High scores more meaningful than low; i.e., you 
deserve the score you get but maybe you should have more 
Factors reducing scores (which are extraneous to the tes 
itself) are far in excess of those raising scores. 


In my estimation a good group intelligence test gives a mod 
erately dependable estimate of the person’s present functions, 
level, but it does not predict his future standing as well as 
like to think. 


: ; t 
The above vote of confidence is predicated on the employer. 
of the best tests; and one long enough to be reliable. 

short tests are not reliable. 


My response here (“moderately dependable”) refers to the 
"best" group tests. The widely used “self-administering 
group tests are “doubtfully dependable.” 


The problem here is complicated by growth effects. À x 
children, destined то be normal, start growing late Abl ii 
stunted. Repeated retesting, graphing the results, enab'e 
this to be detected. 


I believe intelligence test ratings are a little more dependable 


for the older school child (high school) than they are for the 
younger ones. 


Greater dependability with younger school children than with 
adults or older school children. 


The younger the child, the greater the question. 

Varies with the age of the child; very inaccurate in first years 
of life, and not dependable for adolescent years. 

Whatever general mental ability exists is less subject to chang? 
1n most cases after school-leaving than before. 


I am more sure about this (dependability of person's Г 
score) for school children than for adults because children. e 
usually subjected to a similar school environment and abiliti is 
have a more equal chance to develop. With adults 1t 
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possible to have differences occurring because of differences in 
environment—due to country of origin, work, etc.* 


Question 2c 


Are people with limited schooling rated correctly in com- 

Parison with persons who have more schooling, or do the 

former tend to be rated too low? 
Limited schooling not a source of error ...... 0% 
Source of slight error ........... zx 


Source of considerable error . 
Source of serious error 


Number of cases ...... дезе ынаа атры. 
No answer or not classifiable .............. 


Here, too, comparisons have been made in terms of whether 
the respondent checked “clinical work” or “psychometrics” as 
а principal part of his activities. Again, the psychometricians 
&ve slightly more favorable evaluations than the non-psycho- 
Metricians. They give the rating of “slight error” in 39% of 
the answers as compared with 31% for the non-psychometri- 
Clans; "serious error” is given by 3% as against 10% by the 
others. More of the psychometricians proportionately refuse 
to answer the question as asked, however (22% as compared 
with 12%). Clinica] psychologists use both the upper and 
Жы extreme ratings a little more than do non-clinicians 

slight error” 44% versus 30%, “serious error” 12% versus 


3% Of arg 
Ae and are more reluctant to answer (24% Versus BA give 
answers) 


1 8 ; 
istrative Comments of Respondents on Question 2 


that E Comments under this question most frequently ud 
of the E answer turns on the kind of test used or Ка 

test, “st content (i.e., the problem is more serious with gr Shi 
Ones and verbal tests than with individual and xd oe 
Only otp Venty-four of the respondents made this ea 25) 

depend er ideas expressed in many answers were that the r p y 
to 5 Upon how extreme the educational lack; whehe ke 
E 9f educational opportunity or not; and what the 
nt, 
t 


eresti Е г о some 
hosting q six quotations. T 


isagreements occur among these last уюгу of the words 

Tees of апа “old iscrepancies doubtless Sum from the apia find what other 
ot di ег.” Further inquiry wo e 

‘agreement are present. idis 
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nature of the cultural environment of the persons considered, 
apart from their schooling. Examples follow: 


The verbal group tests place a considerable premium upon 
formal education, particularly on the first few grades. Non- 
verbal tests tend to reduce this effect. Recency of education 
is also an important factor. 


Generally, the error is slight if exposure through at least the 
first six grades exists. (Another respondent says: Consider- 
able error if less than 30 months of schooling.) 


Again depends on type of test employed and how cautiously | 
interpreted. Also depends on cultural level of environment 
within which limited schooling obtains. Also depends on pos- 
sibly related language handicap not due to lack of schooling in 
own language. 


This likelihood of error is particularly true for individuals who 
are dull. Unusually able individuals tend to rise above the 
limitations of schooling and to acquire such information by 
other means and from other sources. 


Generally speaking, mental ability determines amount of 
schooling. Bright individuals with limited schooling make 
high scores, whereas dull individuals who have been kept Jn 
school a long time make low scores. 


People with limited schooling rate lower than they will with 
more schooling. ‚ Whether this is an “error” depends on one 5 
concept of intelligence. 


The larger the adequacy of the education, the more accurately 
does the test reflect the true capacity of the individual. 


In my opinion, the test tends to rate correctly those with 
limited schooling who have had ample opportunity for school- 
ing; there is a source of considerable error where the groUP 
with limited schooling are those coming from communities 
where opportunities for schooling have been limited and where 
the limited schooling is the direct result of limited oppor 
tunities. 


Source of slight error for typical American communities; about 
85 to 90% of children. No general statement is adequate; for 
perhaps 5% of our school children the error may be serious; 
and “considerable” for another 10% (especially in certa! 
regions). 


Question 3 


(a) Do you find any serious misunderstandings or false Ka 
pectations about intelligence tests (or “I.Q.” tests) on t 
part of non-psychologists? 


VALUE OF INTELLIGENCE TESTS 


Doubtful or don’t know .. 


Number of cases ............... 
MG SISSE „ъъ oca we otra sae 
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(b) If you do, what are the principal misunderstandings? 


The types of misunderstandings mentioned by the respon- 
dents have been classified as shown in the following list. The 
Items are arranged in descending order according to frequency 
of mention. The percentages are based on the number of 
respondents who named any classifiable misunderstanding, 
namely, 74 persons. Since the parts of a single answer were 
often classified under several headings, the percentages total 


More than 100, 


“Principal Misunderstandings” 


Over-rating the test results; exaggerated belief in test 
validity, reliability, accuracy, constancy of I.Q., etc. 
elief that a test measures all aspects of ability; neglect 
9! separate abilities; use of I.Q. for purposes for which 
Dot intended аен бу рп youn. sa osse Toa, iei 
onfusion in meanings of terms (1.Q., M.A., percentiles, 
intelligence, etc.); thinking of any test rating as an 

‘Q.; confusion of intelligence and information; wrong 
use of I.O. applied to adults ........ ee 

Tendency to go to extremes in appraising tests; they are 
Wonderful or they are worthless ..... аеннан 

Assumption that tests measure innate ability; that they 
are independent of environment ................. 

ther misinterpretations of what the tests measure or 


of i i ИБ jene 55 Лане 5 K алай ® we d 
Fail their limitations ..... T" 


Ure to interpret scores in relation to norms or to 
think in comparative terms; misuse of norms ...... 
Under-rating the test results; exaggerated disbelief in 
passt validity, reliability, еїс.............+.+-.+..- 
ailure to recognize that some tests are better than 
others (group versus individual; limitations of verbal 
tests; joy e шыр MT XO ES 
Too much credence given a single measurement, regard- 
less of how and where test was administered ....... 


58% 


55% 


46% 
30% 
22% 
19% 
16% 
12% 


14% 
10% 
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indicated that the subjects were more efficient in each test when working alongs 
although in the addition test they completed more work with the examiner PE 
Introspection showed the examiner to be the stimulus factor. for these effects. 
Further research is needed, according to the author. Vernon S. Tracht. 


Blain, I. J. “The Rationale of Scientific Selection.” Occupational Psychology, XIX 
(1945), 28-34. i | Е 
This article supplements that of Cockett's, re-emphasizing some of i po 

and introducing others that had not been previously mentioned, all in line wich бе 

application of scientific techniques to the problems of personnel and job-p асе 
in industry. Psychological tests are seen to give the employer a more ohien Ms 

critical means of appraising a person's abilities than application forms, schoo! p 

ports, and interviews, although the latter are recognized as contributing net. 

concerning personality, character, and temperament not as yet accurately 
surable. Vernon S. Tracht. 


Burton, Arthur and Joel, Walther. "Adult Norms for the Watson-Glaser Tests of 
Critical Thinking." Journal of Psychology, XIX (1945), 43-49. TEM 
The Watson-Glaser Tests, Battery I, were administered to 150 applicant um 

civil service positions. Ages ranged from 23 to 72 years, with the mean pr 

median 36.9. Subjects below the median age made significantly higher mean higher 
on all tests. The mean score of those with 2 or more college degrees was ificant. 
than those with 1 or no degree, but the difference was not statistically UE 

Norms as a whole were higher than norms for college seniors. More extensive 1 an 

are needed, and a study of the validity of the tests in selection of professional 

administrative personnel should be made. Lorraine Bouthilet. 


Cockett, R. “The Rationale of Scientific Selection.” Occupational Psychology» 
XIX (1945), 20-27. А S may 
Contrasting the autocratic with the democratic way by which a Boe n 

fully utilize the abilities of its individual members, the author describes the P which 

of “scientific selection.” This involves the organized method of discovering V 

persons possess the skills and capacities necessary for certain kinds of work, апте 

analyzing different jobs to determine what abilities are required for them., Bos 
experimental and statistical methods of securing valid and reliable measuring 


d У К ele- 
struments for this purpose are briefly discussed and the personal or “human 
ment” assessed. Vernon S. Tracht. 


Goldstein, H. “A Malingering Key for Mental Tests.” Psychological Buil 
LXII, 104-118. d 
The malingering key is a scale composed of those items which prove n the 

sensitive in differentiating between simulated malingering and genuine failure Papers 
Army's Visual Classification Test. Tt is applied directly to the original test n" and 
and yields a score based upon the number of discriminating easy items aile o 
the difficult items passed. The key is developed upon the hypothesis that ^n fail 
and malingerers give test patterns differentiable because bona fide failures Ж items 
the harder items, while the malingerers will tend to pass more of the har ipate 
than the bona fide failures. In trials with thousands of cases, the key eim bing 
from seventy-five to ninety per cent of test failures as non-malingerers, thus €” 
examiners to concentrate upon the remaining few and prove whether they 
mentally adequate for service. Elizabeth Bell. 


most 


" for 
Jurgensen, C. E. “Report on the "Classification Inventory; a Personality Test 
Industrial Use.” Journal of Applied Psychology, XXVII (1944), 445 redict- 
. Designed to avoid some of the faults of personality tests now used fore roups 
ing job success in industry, this inventory contains 245 items, comprising 4 Kc red 
of three items each and 55 paired comparison forms, and is intended to be t 
and validated on jobs instead of personality traits. The author asserts ghan and 
may be devised on the basis of traits for utilizing it in industrial, education? g on 
clinical guidance work, but that occupational selection requires keys 
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Specific jobs. Reliability and validity have been “satisfactory” in the two studies 
thus far completed, validation having been on groups comparable to those for whom 
the Classification Inventory is planned. Vernon S. Tracht. 


Lowenfeld, V. “Tests for Visual and Haptical Aptitudes.” American Journal of 
Psychology, LVIIL (1945), 100-111. 

. Having determined that there are two groups of individuals with respect to 
their Orientation: a) those who use their eyes as the main intermediaries for their 
Sense impressions and b) those who, though with normal sight, depend upon touch 
and kinesthesis, the author has developed a battery of tests for use in job selection, 
Where either the presence or absence of these abilities may be significant. On an 
SXberimental group of 1128 reactions, it was found that 47% were clearly visual, 

о Were clearly haptical, and 30% were not distinguishable. Stress on this knowl- 
edge is important in job situations where the presence of one or the other aptitude 
Would be a liability. Francis Medland. 


McHugh, G. “Relationship between the Goodenough Drawing a Man Test and 
the 1937 Revision of the Stanford-Binet Test" Journal of Educational Psy- 
chology, XXXVI (1945), 119-124. 

he author found ап r of .45 between M.A. scores and an 7 of .41 between 

the LO, scores on the Goodenough Drawing a Man test and those on the 1937 

Stanford Revision (Forms L and M) of ninety kindergarten children in the public 

Schools, These 7’s, he believes, may be depressed because of the equal use of both 

orms of the Binet. A further study of the bi-serial 7% between individual scores 

sf the two tests reveals that limiting the Goodenough to the nine items which have 

a biserial correlation coefficient of .30 or better with Binet LQ. produces the best 

is 3tOnship between the two tests. Thus the number of Goodenough items used 

or kindergarten could be much reduced without losing reliability. Elizabeth Bell. 


McNamara, W. J. and Weitzman, E. “The Effect of Choice Placement on the 
ifficulty of Multiple-Choice Questions.” Journal of Educational Psychology, 
‘XVI (1945), 103-113. . Eo 
choi t is generally believed that the "chance" element. in "five-choice m ins 
study objective questions is “опе-іп-буе” and “one-in-four,” respective у. _ This 
one Тү“ ётї1рїз to determine the possibility that placement of the correct hoice in 
iten or the four or буе possible positions has a definite and measurable effect Spon 
on b difficulty, On a group of Naval Cadet subjects the next-to-ti east pud " 
cho; oth four- and five-choice items, had the greatest difficulty level. On M 
AER ltems, the difficulty increased from the first through the third position, n 
a { choice items: the second and third positions were less difficult than Де pt 
indi the fifth position not significantly more difficult than the first. е ata also 
the fate that the act of reading several incorrect choices has little or no е! js upon 
ability of the subject to select the correct choice in the series. Francis Medland. 


Schmia iagnostic Aid: Th 
tH. O, illi F. Y. “Test Profiles as a Diagnostic id: е 
emreuter pe a MUR of Abnormal and Social Psychology, XL 


(1945) 70- 

» 70-76. 

of the C Army psychologists found that when subtests BI-N, pira aid Вр 
сај z Bernreuter Inventory were considered in relationship to each other by sa к 
devi тетеп, they were able to differentiate “standard normals” from mm 2 
Army 5; With an approximately 80% degree of certainty. Enlisted ШЕН a 
the Y Air Forces made up the 2 groups of subjects, 100 in the normal an in 


analadjust i iatric diagnosis by competent medical 
Office, Justed, the latter having had psychi іч и а 


beli vel ile not indica ing the direction of maladjustment, the inventc 
n ; Mar E i i d ti indi iduals. 
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тоу S.T, he authors to be a time-saving 
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states and hysterias, were tested over a two-year period. The main tests were: 
Progressive Matrices, Cattell Па and IB, and the Shipley Vocabulary Test. Those 
with obsessional neuroses tended to be more intelligent than those with the other 
types. There was no proof that the others differed significantly; in other words; 
“heurotics are heterogeneous as regards intelligence.” It is argued that if we fint 
many common characteristics which differentiate neurotics from normal persons, We 
can expect also to find, when we isolate a particular characteristic, that the “neu- 
rotics are heterogeneous in its respect." Elizabeth Bell. 


Staff, Psychological Section, Office of the Surgeon, Headquarters, AAF Training 
Command, Army Air Forces. "Psychological Activities in the Training Com- 
mand of the Army Air Forces.” Psychological Bulletin, XLII (1945), 37-54. 
This is the seventh of a series of articles and is a report of the activities 0 

the Section in the "application and correlation of the various tests used in the dus 

fication of aircrew members" and in the supervision and coordination of psycho 
logical research activities, including test development. Emphasis has been shifted 
from selection of aircrew trainees for success in training, which has been prac 
tically solved, to the selection of those who will make good combat officers. Dat 
are being secured against which the tests may be validated. Efforts are being uie aa 
to select men for the more specialized functions as members of lead crews, te 

pilots, and bomber pilots, and for the various types of gunnery training. Е 

is also an extended search for proficiency criteria in training. Elizabeth Bell. 


» 
Wilson, С. M. and Burgess, Faye. "Construction Puzzle B as an Ability Test. 
Journal of Educational Psychology, XXXVI (1945), 53-60. board 
This is a discussion of the results of using Construction Puzzle B, a Гогт-00' "s 
test, as part of a battery of tests given in a war industry for classification purpos ой 
It was observed that some subjects who were low on the form-board test Moos 
high on other tests. The need for special attention and interpretation, aS Mo ilet 
the need for further study, in regard to this test is emphasized. Lorraine Bout?" 
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SOME CONCEPTS OF JOB FAMILIES AND THEIR 
IMPORTANCE IN PLACEMENT 


HERBERT A. TOOPS: 
Ohio State University 


Ture are, it is asserted, some 30,000 or more occupations. 
hat is a large number. We could hardly construct that many 
aptitude tests, say, in a century. To any other proposal affect- 
Ing all the occupations опе would have to make a similar com- 
ment. The task is too big; it would not get done. 'There is 
Breat need, accordingly, for somehow reducing the number of 
kinds? of occupations. Could one sort out, for example, a 
Small number of type-occupations which would stand for or 


uu the lot of them? 
is hope is analogous with the correspo 
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Ychologists regarding human types. They hope to be able 
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enough persons for a study; enough that any statistical follow- 
up would be statistically reliable. Without some such concep- 
tion as “unique personality profiles” every person obviously 
has a profile so “complex” that he is his own type, thus result- 
ing in 140 million such types in America, and no studies are 
possible. 

Just as the botanists find it useful to group their plants for 
study, and the zoologists their animals, so must psychologists 
find a way to group their humans. How to do this is the ques- 
tion. This they may not do by employing the United States 
Census classifications. The psychological differences, for ex- 
ample, are small between whites and Negroes; between men 
and women; or the single and the married. We must search 
deeper for the more fundamental differences among individuals, 
indeed for the traits which are the “basic dimensions” of 
humans. 

The traits on which unique personality profiles are founded 
must be chosen—it is known—to obey such general prin- 
ciples as: 

1. The traits must measure basic human “dimensions” or 
qualities. Another way of saying the same thing 1s tO 
state that: 

a. The traits, in the aggregate, must correlate highly 
with success in all the important endeavors О 
mankind, particularly the occupational success 
criteria. 

b. They must correlate zero, approximately, with one 
another to the end that each such trait thus mea- 
sures as much as may be something different from 
every other, a basic human “dimension.” 

2. The number of such traits to be measured will therefore 
become a minimum; and the “minimum profile of per- 
sonality” results. 

3. The traits must be objective and quantitative, and 
(preferably) not too expensive of time or resources for 
their accurate measurement; in a word, they must be 
capable of being measured by practical tests or mea- 
suring instruments. 


SSS 
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Obviously analogous concepts underlie any attempt to 
classify occupations. The “traits” of occupations likewise must 
Pass the same standards. They too must be unique, minimal 
in number, and as objective, quantitative and practical as 
Possible. There is here, however, an additional requirement, 
Popularly referred to as a means of “bridging the gap between 
the man and the job.” Briefly this means that, if possible, the 
occupation should be measured in the same units as the man, 
to the end that we may form reasonable judgments of whether 
this given man can do this particular job; or rather, since the 
man is the reference point rather than the job—or popularly, 
since industry exists for man and not man for industry—which 
job, of the myriads of jobs, can a man of this particular profile 
їп general do best? The reference point often is reversed, and 
Particularly in wartime where filling the “openings” is the 
Paramount consideration, even though it does not follow in 
every case that the individual “works at his highest skill.” 
War is an emergency; it is met with emergency behavior. Thus 
wartime we may find it desirable and necessary to employ 


Practically all 24-percentile, or higher, chemists as chemists; 
ait we also may employ even 100-percentile lawyers as clerks 
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customarily he has thought of the job as a different-order-of- 
creation, with its own traits and categories that bear little or 
no relation to human traits and categories. Clearly then any 
mode of classifying occupations and jobs which will emphasize 
such values and relationships—particularly any which will 
make the relationship obvious—will be valuable and likely will 
lead both to better selection (placement) and to better gui- 
dance. Ideally, if there could be a one-to-one correspondence 
between the occupational “traits” and the man traits, place- 
ment would be maximally facilitated. Possible methods of 
making this correspondence are considered herein. Briefly, 
they are: 

1. Stating the requirements of the occupation in terms of 
the average or, preferably perhaps, the 75-percentile 
as-to-success worker therein, on the traits on which it 18 
useful—i.e., valid—to measure the workers. 

2. Specifying an occupation as an idealized set of human 
requirements, or profile of human traits, based on human 
judgments of the occupation’s “requirements.” 7 

3. Specifying all occupations іп terms of the human traits 
which a factor analysis of data, obtained preferably by 
method 1 above, would reveal as the common measures 
of all jobs. 

4. From empirical observations of success on jobs of а very 
large number of men of varying profiles, specify as the 
human-characteristics profile of the job, that human 
profile which succeeds best at the occupation in question. 

The attempt to classify occupations has so far led only 
to the concept of job families. The concept has grown out of 
employment office work, where in normal times they have ОП 
hand unemployed men needing work. Their search at such 
times is for an available job which a given man, say an account- 
ant, can do. Their special knowledge consists in convictions 
of what a given type of man can do. 

Our problem is to increase that knowledge and to amplify 
its dependability. During the depression it became obvious 
that an accountant could do labor work, if he had to, in order 
to live. Even depressions, however, do not greatly increase our 
knowledge of what more complex work accountants can do. 
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‘Clearly, if we could systematically accumulate such experi- 
ence for all or most men of the community as they change jobs, 
we should soon have a vast fund of knowledge for sifting and 
refinement. In wartime many drafted men arbitrarily and by 
chance are placed in occupations new to them, but in most cases 
these too are down-grading; and furthermore, little record is 
made of the satisfactoriness of such placements. At such times 
much job-dilution (job-simplification) is resorted to in an 
effort to employ less capable talent on small units or special- 
ized aspects of more complicated work, to improve production 
and to increase accuracy by the development of great individual 
skill on non-complex tasks. 

It has long been known in employment office work that 
when industry could not get “exactly the type of man it 
Wanted," a man of “certain specific occupations" often might 
do almost as well. In Swan’s Index, for example, prepared for 
the C.C.P, personnel work of World War I, the “substitute 
Occupations, or civilian equivalents," of some 750 Army trades 
and occupations were determined and listed for the use of all 
Personnel officers. The problem, it has been stated, is that of 

Ow to transfer workers from job to job with the fullest pos- 
sible utilization of their previous skills. Thus the Army should 
exploit the civilian occupational skills of the draftee, and in 
turn industry, after peace, should exploit the Army-acquired 
Occupational skills of the demobilizee. Inasmuch as any defi- 
Nition is always defective, let us enumerate the possible char- 
acteristics of a “job family,” by implication, from a considera- 
ton of the actual or proposed practical uses of such “job 
families”; 

1. When up-grading, training will be more effective if it is 
for a new responsibility which falls in the same family of occu- 
Pations. Thus shortages of manpower may be readily filled 
rom excesses of manpower if such exist anywhere in the same 
family of occupations. By such action the amount of re-train- 
ing will be minimized. (It will be recalled that different occu- 
Pations turn out different products and these are peculiarly 
affected, in wartime particularly, by the presence or absence 
of raw materials. Accordingly, “excess” of manpower may 
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crop up almost anywhere after every new governmental order 
or decree.) 

2. For determining substitute occupations for individual em- 
ployment purposes, where the additional training, if any, is to 
be afforded by the job itself, job families provide the necessary 
factual information. The greater job families, based on indus- 
tries rather than occupations, enable wise decisions to be made 
in converting non-essential industries to essential industries, 
with a minimum of waste? The complete present manning 
tables of a factory compared with the proposed manning tables 
of a new production schedule reveal at once the distribution 
of occupations freed and those necessary for operations after 
the conversion. Let the distribution of occupations released be 
abscissae (X) of a two-way table; let the proposed new distri- 
bution be ordinates (Y) of the same table. The conversion 
is effected then in the following series of steps: | 

2(1). Into the diagonals of the table, which have identical 
labels for each coordinate, are placed the several frequencies 
of persons who can be transferred directly without conversion, 
into exactly the same occupations. The personnel cards, ОГ 
duplicates thereof, of these specific persons are then sorted from 
the total work-roster and are filed under the new manning 
categories. 

2(2). In each diagonal compartment, form a ratio of the 
number thus obtained to the total number desired in each of 
the several occupations. Parallel this record with another of 
arbitrary indices of indispensability of the unfilled portions. 
By a study of these two ratios, giving weight to the importance 
of the new occupations in the scheme of production, and to 
salaries, training time, and possibility of obtaining outside 
trained recruits, establish an order for filling the new occupa- 
tions. It would be highly desirable if this angle of the matter 
could be made objective and mathematically determinate. 

2(3). Reduce the X-marginal frequency, or abscissal mar- 
ginal row of the chart, by the frequencies of persons just allo- 
cated (Step 2(1) without conversion). 


_?Shartle, С. L., Dvorak, Beatrice J. and Associates. "Occupational Analysis 
Activities in the War Manpower Commission.” Psychological Bulletin, Vol. +” 
(1943), 703. Shartle, C. L. et al. “Ten Years of Occupational Research." Voc 
tional Guidance Magazine, XXII (1944), 387-448. 
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2(4). Considering now the new occupation (Y) to be first 
brought up to strength: by aid of the job family classification 
pick out the first conversions therein, if any, those which in 
general can be made with a minimum of retraining. . Add the 
frequencies of the corresponding persons to the compartment 
Which is designated by an X of the to-be-invaded occupation 
and a Y of the to-be-established occupation, somewhere be- 
neath the diagonal row of compartments; and remove a corre- 
Sponding number from the X-marginal frequencies and also 
from the remaining roster the cards of those so related and 
Converted. Similarly invade the other earliest-to-be-invaded 
Occupations; and when all is complete establish in a second 

-marginal column the frequency of the most indispensable of 
the new occupations. If the quota is filled, the process is 
Stopped short at that point, only enough persons being taken 
from the last occupation to fill the need. | 

2(5). In similar fashion, consider the second occupation 
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B. Shortages in the proposed new manning tables which 
must be filled by resorting to outside recruitment. 

Clearly there is room here for the development of mathe- 
matical methods of optimalizing placements. : 

3. Transferring an unsuccessful person within a firm into 
an occupation in a job family in which he-had formerly been 
successful, as an alternative to separation from the firm, is wise 
personnel management. 

4. In giving guidance to individuals who wish an improve- 
ment in their situation but are unable to forego an income while 
retraining, job families are helpful. 

It is a well-known principle of adult guidance, for example, 
that if possible the guidee should exploit his most successful 
experiences and training rather than merely abandon them 
when he changes occupation. 

5. In recruitment of personnel for new industries, 
families inventories yield valuable information as to where 
such recruitment will yield the best results at the least cost- 

6. In the assignment of recruits to related military occu- 
pations, job families prepared for the Army, Navy, Marines 
and U. 5. Coast Guard have been helpful.* n 

7. That wages of the member occupations of job families, 
other things being equal, should differ but little, is a principle 
of value in wage adjustment. 

8. The various occupations of job families probably have 
highly similar physical demands, thus making it possible tO 
know alternative positions into which a given handicapPe 
person will fit, without preliminary research or trial and error 

9. In promotions one wants to maintain the principle ? 
individual growth, rather than that of revolution, so that ? 
knowledge of job families should be helpful in establishing 
paths of promotion. One is ready for promotion when one 1$ 
proficient in his present job in all the elements thereof that 216 
common to the next job ahead, this assuming that the paths 
of promotion previously have been established accordin& 
to such principles. One's present job is thus always a produc 
tion field for exercise of the skills, knowledges and tech- 
~~ 8Shartle, C. L. et al, ор. cit., p. 704. 
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niques obtained in the previous position, and a training field 
for acquiring those skills, knowledges and techniques basic to 
the production of the job ahead which also are a part of the 
reasonable practice of the present position. 

10. In aptitude test construction, a test conceivably may 
be devised which will be reasonably adequate for all the occu- 
Pations of a job family, thus restricting and constricting the 
field a great deal. 

11. The reabsorption of veterans and the shift of civilians 
from wartime production to peacetime production after peace 
and the demobilization will involve a stupendous task of 
worker transfer, retaining and allocation. As an aid in this 
task, job families, applied in reverse, will be useful to locate 
the logical job destination of workers in civilian industry when 
war and war industry no longer exist. In theory this is analo- 
gous with the construction of a decoding code. Inasmuch as 
it is reasonable to believe that many demobilized at peace will 

€ ready for promotions, their after-war destiny logically also 
should be a related job-family occupation to which a given 
Worker “should be promoted” in view of his increased "skill" 
acquired during the war years. The longer the war, the less 
advisable it is for a larger and larger number of the de- 
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Occupations grouped on the basis of the work performed 
(i.e., sorted on the basis of action verbs, such as welding, sew- 
ing, nailing, gluing, etc.) seem more alike than if grouped on 
any other basis. In the United States Employment Service 
sorting of the “traits of occupations” to determine the job- 
families, the first four of the above list occur in the order above 
named in the sorting operation. Shartle's results seem to show 
that “people,” rather than specific kinds of people, are required 
for a majority of the occupations of industry, and that the 
training time is, for the great majority, unbelievably short. 
Wartime training methods of unusual potency emphasize the 
correctness of this conclusion. Motivation was often at 2 
maximum. If you did not learn, you got your head shot off! 

As a means of facilitating their use, the coding scheme 
employed to stand for an occupation should take into account 
the family relationship of occupations by ascribing contiguous 
numbers to the highly related occupations. Occupations thus 
break down into “fields of work,” somewhat narrower “process 
groups” and, finally, into still further variations, varieties, 
alphabetically arranged (and distinguished by unit digits 1" 
the USES code number). 

Just as in the taxonomy of botany one may arrive at various 
conceptions of what is a “family” (a sub-division of higher 
divisions of classification) so one may arrive, by different 
routes, at different classification principles, at different aggre- 
gations of occupations which in the several classificatio? 
systems logically may be called families. Some half doze? 
such alternative, actual or potential systems of deriving fam- 
lies of occupations will now be outlined. 

1. The most extensive development of job families has been 
made by the technique developed by the WMC, Worker Anal- 
ysis Section. It involved the following steps: : 

1(1). A job characteristics form, consisting in its final edi- 
tion of 47 human traits, was filled out by from one to fiftee” 
analysts in different parts of the country observing the 
same occupation, as distinguished from job-in-this-particula- 
factory. 

1(2). The same analysts also observed the duties of the 
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worker, the tools used, the equipment, the materials and the 

minimum education and experience necessary for the work. 

The results were systematically recorded in a Job Analysis 

Schedule form. 

_ 1(3). A compromise set of “traits” for the given occupa- 

tion was decided upon, and was entered into a Master Worker 
aracteristics Sheet. 

1(4). The results now were recorded into Speedsort cards 
and other pertinent observations about the occupation were 
Written on the face of the card. 

The cards now represented in highly condensed form the 
Tesults of judgments of what are the important characteristics 
of the Occupation in question. Some 9,000 occupations, in some 

industries, were thus reduced to Speedsort cards, a kind of 
Catalog of occupations which could ultimately, after the job 
18 finished, become the card catalog of American occupations. 

y the aid of the usual sorting needles, presumable families, 
«Упа regard for the above list? of judged most-important 
traits," were sorted out. This was aided by noting that as the 
torting progressed, channels in the edges of the cards began 
to develop on other traits of a family than in those traits actu- 
dy Sorted upon, the traits of the list above. Aided by these, 

s member-occupations of presumable job families were 
Sorted ott: 7 
а The Complete job analysis description was appealed ез as 
` Urther test for excluding some of the presumable members 
fam; š family; and to insure that all that should belong da 

Пу had been included. The job family was now pu 

is^ There was still a need for са туре” 
fr $ of the job family were most like the ideal Jo 
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exceeding the previous “weight” of this particular member- 
occupation of the job family, there was ascertained an aggre- 
gate numerical value (the sum of the points-ratings) by means 
of which the specific occupations could be ranked in a decreas- 
ing order of degree of resemblance to the type job. 

By aid of these, the job-family was split into three sub- 
divisions: 

Closely related occupations—those in which workers of 
experience could probably be transferred to any other 
member-occupation of the closely related group with little 
or no retraining. «di 

Less closely related occupations—those for er 
workers required more retraining to make them accepta s 
workers. : 

Least closely related occupations—those barely meeting 
the minimum requirements for useful similarity, and for 
which workers, although they might require complete re- 
training, nevertheless because of their worker character- 
istics and other considerations of knowledge, skills, 6С 
would be more likely to succeed at the work than just any- 
one chosen at random. 

The apparent strong points of the method are: 


a. A wide variety of “traits” of occupations was inc 
vestigated. Я 

b. The analyses of the job were made by trained an? 
lysts. 


c. Observations of the same occupation were made S 
different parts of the country so that sectional д 
crepancies could be allowed for, even to the «c. 
of splitting up an ostensible occupation into two 0 
more; or it could be noted that occupations bearing 
differing names in different parts of the country 2 
in reality one occupation, not several. уе 

d. A common denominator of the several alternat! 
analyses was decided upon. "s 

e. The job families resulting had other than statisti i 
and logical definition and justification, since the sor 2 
ing of the Speedsort cards was augmented by 
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critical scanning of the entire description of the con- 
stituent occupations before adoption. 


Its more obvious weaknesses are: 


a. 


b. 


The analyses were subjective, even though done 
after a uniform schedule or outline. 

The list of job “traits” was not wide enough to cover 
adequately the professions. 

The comparison of the reports of the as many as 
fifteen analyses of a given occupation was done 
centrally by a person out of touch with the field and 
without giving the analysts who did the work a 
chance to correct the most obvious misconceptions. 
The completed analysis was not generally sent to 
them for criticism before official adoption, although 
some one or more analysts scanned each of the pro- 
posed analyses before adoption. 

Different Speedsort operators, with the same end 
in view, presumably could come up with different 
potential job family-occupations. In other words, 
the system was not fully statistically determinate. 
There is some doubt whether the skills and traits 
of workers are in every instance transferable merely 
because the pattern in the Speedsort cards is highly 
similar. A watch-maker and a cannon-barrel borer 
might come out in the same job family and on the 
basis of the statistics have every warrant for be- 
longing to the same family, yet the psychological 
characteristics, particularly as to precision, may be 
quite different so that actually there is little trans- 
ferability of skills. 

The double dose of subjectivity involved in putting 
the sub-members of a job family in decreasing order 
of resemblance to the common type is unfortunate 
when there exist several alternative techniques, 
independent of subjectivity, which will give a nu- 
merical measure of the degree of correspondence of : 
two profiles. The ideal jobs or type jobs we take to 
be a pure figment of the imagination. It has no 
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more objectivity than anyone’s conception of the 
“ideal mammal,” the “ideal liliacea” or any other 
taxonomic classification. In fact it does not have 
the rigidity of definition of a sub-classification of 
botany in that all liliacea are spermatophyta (seed- 
bearing, with true roots, stems and leaves), anglo- 
sperms (with true flowers, each containing, nor- 
mally, four whorls of floral organs), autophytic 
(possessed of chlorophyll), monocotyledonous (one 
cotyledon in each seed and with parallel-veine 
leaves) and bulbous (having bulbs). 

2. Inverse factor analysis may be employed as an alterna- 
tive means of ascertaining a job family. If the characteristic? 
of an occupation can be quantified and measured, a factor 
analysis—occupations replacing the usual human names—will 
reveal what factors of job-traits underlie occupations in genera “ 
In this there is no necessary restriction of “the occupation TO 
any particular pattern of measurements, for these may be quite 
as broad as, or broader, than the WMC list above; and in the 
case of the professions’ almost surely will be broader than that 
list! Occupations having similar factor loadings belong to the 
same family. 

3. A variant on method 2 above emphasizes succes 
plying of an occupation and implicitly recognizes that a“ 
occupation has at least a fringe of people “who ought not to 
in the occupation.” If the various X’s of the above metho 
are replaced with a sufficiently varied list of the human traits 
of, say, “successful workers” of the occupations in question 
the resulting factors will discover the basic human factors " 
workers. In this case also one may express the occupation FA 
terms of its component factors and factor loadings. The ue 
ings, here, are the profile of the occupation; and those wor 
tions with highly similar profiles are jobs of highly sim! ch 
human requirements. The occupations which resemble 4 
other most comprise the job families. 


sful 


€ Professions are generally characterized by a higher level of job-traits, +, 25 
characterizes occupations which are not professions; in particular in such resh ly n 
research, administration and supervision, and knowledge. There is proba {опа} 
job-trait found uniquely among professions save that of being dubbed a “profess! 
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The occupations of a given job family presumably may be 
located after the occupational profile is obtained by such 
methods as 1 or 2, either by (1) the Sagebeer Index, or alter- 
natively by (2) the Stephenson inverse factor method. In 
employing the Sagebeer Index, a rationale of research for 
locating families would need to be developed. 

4. A fourth method might be dubbed the universal follow- 
up method. If one kept track of all or nearly all placements 
of professional personnel and at the conclusion (resignation, 
discharge, transfer) of the jobs of such personnel collected a 
simple verdict as to whether or not the person in question had 
been “satisfactory” in the position in question, one might 
analyze the resulting data to note by a simple correlation index 
which occupational transfers are successful and which are not. 
With all the occupations of concern appearing in alphabetical 
order as both the ordinates and the abscissae of a two-way 
table, one may let Y be prior job and X be subsequent job. 
In the compartments thereof, let four-fold correlation coefh- 
cients, solved by the formula, 

ad-bc 
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The letters of the diagram (Fig. 1) are the frequencies of 
the table, here recorded literally to show the mode of compu- 
tation by the formula. From such a table of r-indices, by 
means of a suitable index to be devised, one could readily sort 
out the job families. It will be noted that one obtains here 
two-directional indices. Thus, not only would one consider 
the question “Are the failures of job A also failures when they 
subsequently enter B?" but also “Are the failures of B failures 
when they subsequently enter A?” If the two tendencies are 
vastly different, as in some cases they may be, we might dis- 
cover here some preferred sequences of progression in acquiring 
vocational skills. 

It is normal for wide-awake, alert human beings to take on 
more and more skills, including jobs and even occupations, 
as they develop and mature. We know little enough about the 
more complex end patterns characteristic of “the expert. 
One interesting question here is whether the expert usually 1$ 
characterized by having acquired his complement of skills in 
an orderly progression. If the answer were in the affirmativ® 
obviously one might similarly order the acquirement of skills 
of all people to very good advantage. This method logically 
requires that all the experience of a large community of occu- 
pations be amalgamated, say of a WMC area at least. 

5. The logical or evolutionary concept of job families, 
scribed by Knott,’ conceives that jobs which belong to a family 
are the variations which a specialized industry has evolved out 
of a simpler craft. Thus hat-making, cap-making, dressmakin& 
and even baggage-making in common employ variants of the 
power sewing machine (both the tool and the corresponding 
skills), itself an invention superseding in turn hand-sewin& 
And typists, stenographers and secretaries are variants of the 
amanuensis or letter-writer who still plies his skills in countries 
where most of the inhabitants are illiterate. In such varia? 
some retraining is needed before an individual successful in ofr 
mode of sewing may become proficient in another, but, obvi 
ously, the retraining period would likely be much less than 


de- 


1 Knott, Edward E. “Job : ЖО xen Employ” 
ment Service, April, 1941, an Occupational Analyses.” Missouri State 
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a highly skilled operator in any one of such were to be trained 
into a highly skilled operator of, in general, any other job not 
included in the job family, say a lathe operator. Thus an index 
of what the educational psychologists call “transfer value” 
would seem to be at the heart of this concept, which is based 
on logical, that is economic, historic, or evolutionary, concepts 
of the mode of development of the occupation in question. 
Insofar as such families can be ascertained without the neces- 
sity of the empirical evidence of the #4 method above— 
empirical ascertainment of transferability?—the method serves 
usefully to restrict the field of inquiry but does not eliminate 
the need for the previously mentioned modes of study. There 
are sub-families within families and sub-sub-families within 
Sub-families. These may be ascertained more surely by sta- 
tistical analysis than by unaided observation. One strength 
of the method is that more than the one source of data, the 
Customary field observation of jobs by trained workers, is 
insisted upon as necessary to a proper classification. (To 
Properly classify plants, one must study paleobotany as well 
as botany.) With a set-up such as No. 1 or No. 2 above, the 
Sub-occupations, members of eventual potential sub-family 
groupings, would be the Y's of a basic data table whereas the 

’s might be either (a) tools employed, (b) ans aa 
(а oed, Ce) motions, in the Gilbreth sense, employed 
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asking them to give their judgment as to the degree of trans- 
ferability—on a scale: 


1. Considerable (Less than six months additional train- 


transferability ing required ) | 
2. Moderate (Six to twelve months additional train- 
transferability ing required ) А 
3. Slight (Over twelve months additional train- 
transferability ing required ) 
4. No (Transferability unfeasible, since little 
transferability or no training time would be saved) 


—of people in a given series of occupational specialties of, say: 
civil engineer, when and if these were to enter the break-down 
specialties of the same occupation or the specialties of another 
industry, say the engineering specialties of municipalities. The 
present hypothetical occupations of the to-be-released workers 
are Y-ordinates of a two-way table, while the X-abscissae are 
the jobs of an engineering sort now maintained in municipali- 
ties. In this case, hydraulic engineers, for example, were asked 
to judge on the above scale the transferability of each of a 
number of kinds of civil engineers to the several other special- 
ties of the same occupation. The validity of the method hinges 
on the ability-as-judges of the addressees, i.e., upon such factors 
as the extensiveness of their actual knowledge and experience 
with the implied situations judged, the criticalness of their 
judgment, and their willingness to take the necessary time— 
all of which could no doubt be brought to a state of excellence 
by well-known techniques, such as that of using lesser experts 
to pick the greater, and the like. The method has the merit 
of quickness, low cost, and independent verification of the 
observations exploited. It has all the weaknesses of using 
untrained judges in what is essentially a psychological expe!” 
ment. It may be pointed out that the WMC job analysts also 
employed judgment, of small elements of an occupation #0 
establish a profile from which, by statistical manipulation; the 
job family ultimately was determined and the transferability. 
of skills was inferred. Here, however, the transferability ° 
the specialty as a whole is judged. By abstract psychologic? 
principles the judgment required here is easier to obtain bU 


CONCEPT OF JOB FAMILIES 213 


is of lesser validity than judgment of the more specialized 
Sort obtained in the former studies. For the aid of demobili- 
zation counsellors, the less costly method yields results which 
presumably are about as fine as demanded for the purpose. 
The finer break-down of the WMC job analysis obviously en- 
ables more uses to be made of the results, such as advising the 
demobilizee to know what specific education or training he 
should acquire for a better all-round fitness in his new vocation. 
This the Roster technique, with its present paucity of data on 
the individual profession, cannot supply. However, having 
helped the demobilizee to choose wisely a new occupation 
which will exploit his army and pre-army training, such addi- 
tional (educational) information in many cases may be had 
€ven more usefully from a new counsellor, perhaps a dean or 
Secretary of an engineering college. The advisee is less con- 
demned by receiving only parts of the truth than by receiving 
mistruths. If the validity of the method is equal to or only 
slightly inferior to the more laborious method, obviously then, 
on this score alone, it has great merit. Incidentally the statis- 
tics of ratings, for subsequent treatment of the results, are 
already well worked out. The necessary statistical indices in 
NO case are an insuperable element of the whole. The rating 
Scheme for rating officers’ traits in World War I was deemed 
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The common purpose of job families, if the multitude of 
purposes enumerated above can be subsumed under one gen- 
eral statement, is that of ascertaining the paths of transfer- 
ability of occupations in accordance with the need for the 
minimization of retraining. It is by this criterion then, if we 
are willing to accept it, that the competing methods must be 
judged. 

The above statement implies a mathematical function 
which may be minimized (or maximized, according to its set- 
up), a matter which is at once both the delight and the despair 
of the statistician. Suffice it to say, at this stage, that the best 
elements of all the methods likely will yield a better result than 
blind adherence to any one alone. In addition it must be 
pointed out that the statistically most elaborate system, the 
WMC system, was worked out primarily for the list of occupa- 
tions job-analyzed in the course of the field observations which 
were paralleled by the development of the Occupational Dic- 
tionary. Only a very few professions were so analyzed. The 
implications of this will need to be thought through in arriving 
at a conclusion as to the potential values of this interesting 
new “measurement” device. Only a complete analysis of the 
entire problem, including the method of “bridging the gaP 
between man and job’? (the coding, Hollerith sorting, ап 
placement philosophy and practice) will reveal what elements; 
if any, are worthy of adoption. This statement would empha- 
size highly" also capacities and training (or education) to the 
point of specially mentioning them as well as skills. 

It has been suggested that the study of occupations is a new 
social science.” Statisticians have long known that occupations 
correctly ascertained—which the United States Census does 
not even pretend to do—is probably the most basic classifica" 
tion of mankind; of greater human concern, generally speakin& 
than race, color, sex, marital condition, education, age or plac? 


10 H . - . 1 p^ 
dd One simple solution to this problem is to say, "This job requires a тап 
grind valves; this man can grind valves." 


11 Shartle, C. L. et al. “Establishi Famili ait Vocational 
Guidance Magazine, V - ishing Families of Occupations. 
12 Kiso e ol. XXI (1944), 405-414. xxii 


(1944) от. 0. “Occupationology—A New Science.” Occupations, 
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of residence. These categories have the force of tradition be- 
hind them; but they are not as basic, psychologically, sociologi- 
cally or economically as is vocation. Economically all other 
human values directly or indirectly stem out of or are obligated 
to vocation. In correctly classifying occupations, then, we are 
making possible not only accurate studies of this all-important 
institution but also are laying the foundation for gaining a 
control of such matters as guidance, promotion, transfer, and 
the conservation of human talent generally. А statement 
about one aspect of the matter will make clear the importance 
of the matter. If a worker before changing his occupation 
knew with a high degree of certainty whether a proposed 
change likely (ie. on the average) would result in greater 
opportunity for him, or the converse, much human loss and 
unhappiness could be prevented. Again, assuming a compa- 
rable development in human traits, if one knew what was the 
actual distribution of wages in two occupations for people of 
identical trait profiles, the present unjustified discrepancies 
would tend, without artificial restraints, shortly to disappear. 
And if, finally, a similar distribution of happiness-scores (on 
a test of satisfactions, often called morale in the Army and 
industry) could also be attached to the individual profiles in 
the above table any individual knowing his own profile could 
choose his occupation so as to meet his need, whether for a 
greater income, greater happiness or satisfaction, or both. 

The lower the prestige of one’s occupation, in a scale of 
Prestige values, the more a man expects, normally, to get his 
satisfactions outside his work. In the professions one’s work 
frequently is both one’s vocation and one’s avocation and 
recreation. To maximize the individual's salary and job- 
Originated satisfactions is one of the crucial steps in the maxi- 
Mization of manpower. 
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TESTING FOR ADMINISTRATIVE AND SUPER- 
VISORY POSITIONS 


MILTON MANDELL 
United States Civil Service Commission, Washington, D. C. 


. Tur field of testing for administrative and supervisory posi- 
tions 15 one which is approached by many psychometricians 
With a feeling of defeatism. A number of studies have been 
made to determine which testing methods will improve the 
Selection of administrators and supervisors but, in general, 
PSychometricians have tended to stay away from this phase 
of testing. The reasons for this situation are well summarized 


1 
n Dr. І, L. Thurstone’s statement: 


" The intellectual and temperamental qualities that e 
tha 555 in administrative work are probably more complex 
ап almost any other group of abilities that can be thought 


Woul Sychologists who investigate fundamental human Шр 
uld undoubtedly seek to investigate first those traits which 


© assumed to be less complex.’ un 
j though the complexity of the task of experimenting in this 
and wj "08пілей, there still exists in industry, in the Army 
for вир У› апа in government, the problem of Es ae 
force Pervisory and administrative positions. With a i 
> oth civilian and military, of about 60,000,000 in js 
т tn roughly estimate the existence of about 2,000,0 


Supe i 

TVis as : m as been little 

Study СУ Ог administrative positions. [мен © сас 
о 


u 

Position the validity of present selection met cp rin 

in 5, but the great interest of high аали ` okei 

ао 98 selection devices is some indication О the 
techniques in use at the present time. . Р 

adn: е following are general definitions of supervisory a 


mi : А ition 1 апї 
~ trative positions. By a supervisory pan ие 


ip uum — 
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one which involves responsibility for the working conduct of, 
and the quality and quantity of work produced by, one or more 
subordinates. Ву an administrative position is meant one 
which involves extensive responsibilities for planning, o 
izing, directing, staffing, budgeting, and coordinating the wor 
of an organization or part of an organization. The pau 
trator in some cases performs no supervisory duties since e 
may act only in an advisory capacity, but he usually has the 
functions of a supervisor in addition to his administrative 
duties. А 

Supervisory positions can generally be classified on the KU 
of three factors: (a) the number of employees supervised, ( ) 
the nature of the work or the occupation supervised, € i 
the supervisor's level in the organization. It is peint 2 
present the hypothesis that the skills and abilities MES s 
supervise a few employees are different in degree, and per P 
in kind, from those required to supervise hundreds of wes 
ployees. For example, the supervisor who likes to check са a 
fully and in detail the work of a few subordinates may е 
successful, but this same action would probably make i 
bottle-neck if he applied the same method to the проту А 
of a hundred employees. Observation indicates that се. 
method is as much a trait of the individual as it is a functio 
of the position that he occupies. . „ай 

The nature of the work ог occupation supervised 18 he 
important distinction between supervisory positions. A 
methods required to supervise a gang of ditch diggers jm à 
fully differ from the methods required to supervise a ye. 
skilled psychometricians. Some supervisors who have Rt А 
ferred from organizations where strict disciplinary met the 
were in use have failed miserably in attempting to apply E. 
same methods on different occupational groups or in differ 
work situations. P. 

The trend in the field of administration has been to «ne 
sider administration as an occupation per se and to emph ав 
the common elements among such positions. This tre”! ni 
been especially strong in the field of public administratio; 
is noticeable in Army and Navy administration, and it }§ 
dent in the field of business administration. 
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Those opposing this trend claim that knowledge of the 
business or subject-matter of the organization that is ad- 
ministered is also important. Agreeing for the moment that 
administrative skill is more important than subject-matter 
knowledge, however, one can still classify administrative jobs 
into a number of categories on the basis of other considera- 
tions. These categories can be considered to be: (a) positions 
whose problems are of an international or national rather than 
local character; (b) positions involving the “selling” of the 
goals and objectives of the organization to an indifferent or 
even hostile clientele; (c) positions of an advisory rather than 
operating character; (d) positions requiring the ability to ad- 
minister large and complex functions; (e) positions involving 
swiftly moving problems as contrasted with situations where 
Speed is not so essential; and (f) positions involving the ad- 
ministration of a large organization with hundreds of thousands 
of employees as distinguished from a small organization with a 
few hundred employees.” 

The above categories have been presented on the assump- 
tion that some of the skills and abilities necessary for successful 
Performance vary according to these categories. Actually, 
these classifications are based on job analysis and it may be 
found that, in terms of testing, other categories may appear to 
be more significant.? A 


Qualifications 


Before discussing actual results obtained in testing for 
Supervisory and administrative positions, it may be worth while 
to present various opinions on the qualities necessary for suc- 
cessful performance in these positions. Dr. W. V. Bingham 
lists a large number of such qualities: the administrator doesn’t 
eS 


? These differences among supervisory and administrative positions, which are in- 


tended to be illustrative, indicate that thorough job analysis is extremely essential in 
testing work in this area and that the psychometrician should probably have the 
assistance of а person trained in administration to help him Шешу these ген, 
„С. Alexander Н. Leighton. The Governing, of Men. dy cf a W Red 

; Diversity Press, 1945, Leighton, who is a psychiatrist, in his sudy аата 
Versu ‘uthority center, divided. administrators oe Shee ro groups is based 
p, Slereotype-minded." His basic distinction uman values or on adherence to 
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go off *half-cocked," makes consistent decisions, has well- 
thought out policies, obtains real staff participation in the 
formulation of policies, builds a team, has ideas, listens to new 
ideas, is not an egotist, gives recognition for good work, dele- 
gates responsibility, sees his staff, has good timing in making 
decisions, and knows outside conditions. Johnson O'Connor 
reports that successful executives have a large vocabulary, а 
wide range of aptitudes, ап objective personality, an accounting 
aptitude, and an aptitude for their first position.® 

Schell lists as the outstanding requirements for an ехеси- 
tive an innate interest in and affection for people, an outstand- 
ing personality, and a scientific mind.^ Ordway Tead, who has 
contributed extensively to the theory and practice of admuinis- 
tration, has suggested the following qualities as desirable for 
leaders: physical and nervous energy, a sense of purpose and 
direction, enthusiasm, friendliness and affection, integrity, tech- 
nical mastery, decisiveness, intelligence in many directions, 
and teaching skill? In a pamphlet published in 1923, the 
American Management Association recommended the follow- 
ing qualifications and values in selecting supervisors: per- 
sonal—30%; mental—4575; moral—15%; and physical— 
1075.5. Cleeton and Mason offer as their criterion of executive 
ability “above average ability in a large number of qualities 
which can be rated or measured." @- 

It is apparent from this listing of qualities needed by 4 
ministrators and supervisors that major stress is placed ee 
aspects of personality. In addition, however, a number of вше 
abilities are considered essential. While few would require p 
the administrator or supervisor should have the highest xm 
ability within the group that he heads, superior mental abili 


Е for 
+W. V. Bingham. Administrative Ability. Washington, D. C.: Societ 


Personnel Administration, April, 1939. ‚ Steven? 
5 Johnson O'Connor. Characteristics of Successful Executives. Hoboken: 
Institute of Technology, 1932. Graw 


. 9 Erwin H. Schell. The Technique of Executive Control. New York: Me! 
Hill, 1930. e; 1922: 
“Ordway Tead. The Art of Leadership. New York: Whittlesey House. осів" 
5 tene the Supervisory Forces. New York: American Management 
ion, 4 
? Glen U. Cleeton and Charles W. Mason. Executive Ability: It 
and Development. Yellow Springs: Antioch Press, 1934, p. 12. 
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1S still considered very important.” Part of the differences of 
Opinion on this subject may be accounted for by the failure to 
relate the mental ability required to the group being super- 
vised. The mental ability required of a foreman of laborers 
18 obviously lower than that required of a Construction Super- 
intendent but in relation to the mental ability of his laborers, 
the foreman should rank high. 

The administrator and supervisor must be able to under- 
Stand organization. Some technically minded persons are con- 
Stantly frustrated by administrative procedures and regula- 
tions when they are promoted to supervisory positions, while 
others acquire a knowledge of how to get things done which is 
кот to success, especially in a large organization. Without 
fan evidence it is impossible to state what factors make for 
any in this particular phase of administrative күзе уеї 
ad a speculate that this ability is non-intellectual ut re- 

to the interests and personality of the person. 


Use of Paper-and-Pencil Tests 


1. Interest ] nventories, From the published studies, there 
cme indication that the measurement of interests ме 
тенч, substantially to the selection of administrative 

isory personnel? The major problem is in determining 

at norms to use and which special interests, in addition to 

н Interest or interest in people, are desirable for the par- 
аг group of positions being studied. 

п his Study of Federal government administrators, Thur- 
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stone found that the Social Scale of the Allport-Vernon Scale 
of Values differentiated among his population better than any 
other measuring device he used. The Theoretical Scale of 
the Scale of Values also differentiated positively and signifi- 
cantly. The Commercial Interests Scale of Thurstone’s 
Vocational Interest Schedule and the Religious Scale of the 
Allport-Vernon also differentiated significantly but negatively. 

Achard and Clarke, in their study of 300 supervisors in the 
Consolidated Edison Company of New York, prepared а 
special rating scale of the Strong Vocational Interest Blank 
for Men (Revised) and obtained satisfactory results with four 
different types of supervisors. They state that “it was the best 
single all-around indicator of supervisory ability." 

Strong has prepared an occupational interest scale for pub- 
lic administrators based on his work with the Forest Service 
and the Committee on Public Administration. The U. 5. Civil 
Service Commission, in its test development work in connection 
with the administrative intern program, has obtained promis" 
ing results with the Kuder Preference Record. А 

The early studies in this field and the recent investigations 
by Strong, Thurstone, and Achard and Clarke indicate that 
studies of interest, if designed for the particular organization? 
situation, can contribute significantly to administrative ап 
supervisory selection programs. In using interest inventories 
it will probably be found that the appropriate critical scores 
vary significantly among different types of organizations a” 
at different levels of the organization's hierarchy. 7 

2. Personality Inventories. These inventories can be di- 
vided into two groups; namely, those of the omnibus type 
which furnish several measurements for the individual, 506 
as the Bernreuter, and those furnishing a single score, SUC 
as Laird's scale on extroversion and introversion. While every” 
one is agreed that, generally speaking, the personality of us 
administrator or supervisor is probably the most significa 
single factor in contributing to successful performance; ¿t 
information now available would seem to indicate that the 


14 Op. cit., pp. 142-145. 
35 Op. cit, p. 362. 
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present personality inventories, either for reasons intrinsic or 
external to them, are only slightly useful for this testing pur- 
pose and are substantially less so than the interest inventories. 
To what extent this result may be produced by the “fudging” 
of responses is not known. It would seem that a sophisticated 
group of supervisors and administrators in a competitive situ- 
ation would be able to “beat” these tests. 

Achard and Clarke obtained promising results with the 
Bernreuter, scaled in accordance with their own methods.” 
Their scale differentiated the good from the poor supervisors 
їп each of the four groups they were studying better than any 
other test they used except the interest inventory. 

Beckman and Levine obtained favorable results with the 
Allport's A-S test on a group of supervisory employees of the 
City of Cincinnati. In а monograph on his work with several 
industrial firms, Hersey reports the use of personality inven- 
tories of the omnibus type for supervisory selection. Kings- 
bury comments in regard to the use of extroversion-intro- 
Version tests for this purpose that "since the concepts of 
introversion and extroversion are so ambiguous, and the 
Various tests intercorrelate so poorly, further analysis and 
experiment are needed before tests of this type can be accepted 
With confidence for this purpose." Personality inventories 
have frequently been used, with some promising results, for 
comparing leaders with non-leaders in educational institutions. 

oung and Cooper found, for example, in their study of ele- 
mentary school children in the 5th through the 8th grades, that 
the leaders could be characterized as self-sufficient extroverts 
While neither physical nor mental characteristics or interests 
differentiated the leaders from the non-leaders.”° 


Basing a conclusion on the studies made, it is evident that 
_ 


. 25 Jurgensen’s use of paired-comparison and rank-order techniques in his ex- 
peumental form of a personality inventory may contribute to a substantial reduction 
G this factor. See C. E. Jurgensen, “Report on the Classifications Inventory,” 

KA Applied Psychology, XXVIII (1944), 445-460. 
42 cit., р. 362. 
y 18 “Selecting Executives: An Evaluation of Three Tests.” Personnel Journal, 
ПІ (1930), 415-420, 
2o 02- cit, p. 127. 
20“Some Factors Associated with Popularity.” Journal of Educational Psy- 


chology, XXXV (1944), 513-535. 
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the present personality inventories can contribute only slightly 
to the selection of administrators and supervisors and that this 
contribution will be further reduced by the fact that candidates 
may “fudge” in a competitive situation. The factors that these 
inventories attempt to measure are so important for successful 
performance that much further work in this area is definitely 
needed. 

3. Mental Abilities. The contribution of this type of test 
to predictions of supervisory success runs the gamut from ex- 
treme importance to only slight or no importance. This result 
is related to the previous discussion where the opinion was 
expressed that one cannot treat these positions as though they 
were identical, and the difference in the results obtained 18 
undoubtedly the consequence of differences in job-content plus 
biases among raters in the organizations being studied. : 

The results of the Army's officer selection program indi- 
cate a fairly high positive correlation between the successful 
completion of officer candidate schools and general classifica- 
tion test scores. Thurstone in his study of Federal adminis- 
trators found that the linguistics section of the American Coun- 
cil on Education Psychological Examination made a significant 
and positive differentiation between the better and the poorer 
administrators. Shuman in his study of 99 foremen obtaine 
a correlation of +.39 + .07 between the Otis Q. S. Mental 
Ability Beta and performance ratings?  Uhrbrock and Rich- 
ardson, in their thorough study of factory supervisors, foun 
mental ability items useful for differentiating between 80% 
and poor supervisors? Achard and Clarke found that the ОМ” 
Self-Administering Test of Mental Ability, Higher Examine 
tion, was of value for each of the four groups of supervisor! 
studied. i 

The above brief summary indicates that a mental ability 
test should be included in testing programs for supervisory er 
administrative positions. The weight given to the test shoul 


differ, however, depending upon the type of positions being 
studied. 


- : А а 
В 21 "The Value of Aptitude Tests for Factory Workers in the Aircraft Engine en 
“ee Industries.’ Journal of Applied Psychology, XXIX (1945), 159. 
tem Analysis: The Basis for Constructing a Test for Forecasting SUP 


ervisorY 
Ability.” Personnel Journal, XII (1933), 141-154. 
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4. Special Types of Paper-and-Pencil Tests. Mechanical 
aptitude tests have frequently been used, and often with satis- 
factory results, for the selection of factory supervisors. But 
the reason for this successful use has not been fully explained. 
The explanation might lie in the area of the relationship be- 
tween aptitude and interests, with the successful factory super- 
visor being a person whose aptitude and interests are directed 
towards shop and mechanical situations. Whatever the reason 
шау be, however, these tests, especially the Bennett Mechan- 
ical Comprehension Test, have proved useful.** 

In preliminary studies of the U. S. Forest Service, the 
Interpretation of Data Test of the Progressive Education As- 
sociation has given some promising results for the selection of 
administrators. In various experimental studies of this test 
on administrative personnel, the candidates have expressed a 
high opinion of its value. 

Uhrbrock and Richardson, in their previously cited study, 
found that test items on the policies and organization of the 
company in which the supervisors were employed made a sig- 
nificant contribution to the selection of supervisors. It would 
be our hypothesis that this type of test has value in selecting 
administrative or supervisory personnel from among tech- 
nicians and mechanics because it would identify those persons 
whose interests lie beyond the technical parts of the positions 
they are occupying. 

A device sometimes used for the selection of supervisors is 
the multiple-choice or true-false form for measuring the ability 
of a candidate to get the right answer to questions on super- 
visory situations. Quentin W. File's test on How Supervise? 
15 one form of this type of test. Hersey has also used a similar 
type of test. It would seem that this type of test measures 
only the logical factor, and not the emotional factor, in human 
behavior. In other words, a supervisor may know exactly how 
to act in a given situation but his emotions may lead to a 


his statement is not meant to deny that 


different procedure. T an 
this type of test has value but that its primary purpose should 


fu— 


23 See the previously cited studies of Shuman and Achard and Clarke. 
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probably be to determine the need for training. Its value for 
selection purposes has not as yet been determined. - 
Thurstone in his study of Federal administrators obtaine 
significant differences between the good and poor groups on 
three special types of paper-and-pencil tests, Estimating, Clas- 
sification, and Gottschaldt figures. The Estimating Test re- 
quires the candidate, by using logical methods rather than 
knowledge, to arrive at a statistic. For example, a candidate 
might be asked the number of telephones in use for non- 
commercial purposes in the United States for the year 1941. 
It would be highly improbable that any candidate would know 
the answer to this question. He would have to rely on ee 
mating to arrive at his answer. In the Classification Test, bs 
candidate is asked to sort a number of cards, each containing 
the name of a prominent individual, into as many groups ae 
he wishes. Thurstone found that the better administrator 
used fewer categories. The Gottschaldt Figures Test 1s 10 
multiple-choice form and requires the candidate to identify m 
which one of five large, complicated diagrams there к= 
smaller figure. The results obtained by Thurstone justly 
further study of the value of these tests for selecting adminis 
trative and supervisory personnel, 


Oral Interviews 


At some stage in the selection of supervisory and a 
trative personnel, an interview js practically always used. p 
there is no evidence available as toits value. The Army Ser 
vice Forces is now engaged in a program which indicates, 1 е 
least tentative form, that by refining the questions and -— 
forms used, moderate validity is obtained for this test!” 
device. da 

The British Army has been using what might be calle b 
group interview in which the candidates are observed in ae 
discussion of a subject selected either by or for them. Ка 
observation of this type of interview it would seem that t a 
method, when properly administered, offers valuable inform 
tion for administrative and supervisory positions. ‘on 

One of the most common difficulties with the administra? 
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of an oral interview is the need for careful selection and train- 
ing of raters. Quite often the raters are selected on a chance 
basis rather than because of their technical competency. Too 
frequently, also, the training of the raters before the interview 
has been inadequate in terms of the complexity of the task 
assigned to them. 


Ratings in Training Courses 


In those organizations which conduct training programs 
for administrative and supervisory personnel, objective ratings 
on performance in these programs should furnish valuable addi- 
tional information for selection for promotion. The standard 
conditions under which these training programs can be given 
would be a definite factor in helping to make these ratings 
valuable for this purpose. In the promotion examination pro- 
gram adopted recently for shop supervisory positions in Navy 
field establishments, a rating in the work improvement program 
is given a weight in determining rank on the eligible register. 
This examining program is under the direction of the U. S. 
Civil Service Commission, and includes, in addition to the work 
improvement program rating, a written test on administration, 
an oral interview, an evaluation of experience and a perform- 
ance rating. 


Evaluation of Biographical Data 


Public personnel agencies use extensively the evaluation of 
a candidate’s background, in terms of experience and training, 
asa method for rating applicants for administrative and super- 
Visory positions. To the best of our knowledge, there is no 
information available as to the validity of this test for this 
type of position. If this testing method is based on much more 
extensive information than is usually contained on an applica- 
tion blank, it should provide a sound basis for selection if, in 
addition, improvements can be devised in the method for trans- 
lating this information into ratings. Intensive study of the 
validity of this testing method is needed. 

Basic to the success of any study of the value of tests is the 
preparation of a valid criterion. This problem is especially 
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complex in evaluating administrative and supervisory posi- 
tions. Practically speaking, in devising tests for one organiza- 
tion, an attempt at agreement within the organization should 
be made as to the standards to be used in rating. This recog- 
nizes that different standards in use by another organization 
will invalidate the results obtained in the first instance. It can 
be pointed out that in the most successful studies in this field 
as much time has been spent in obtaining good ratings as ОП 
the testing program itself, 


Conclusions 


The conclusions we would draw from this summary of 
trends in testing for administrative and supervisory pOSl- 
tions are: 3. 

(1) The testing program should include a mental ability 
test and an interest inventory. . 

(2) Further study is needed of the value of rape 
inventories, tests of company policies and organization, an 
special types of tests such as the Interpretation of Data an 
the Thurstone Estimating Test. i 

(3) The value of ratings in training courses and on bio- 
graphical data should be explored. 

(4) Precise analysis of the jobs being studied is an absolute 
need in order to identify homogeneous sub-groups. be 
(5) Differences in the value of particular tests should 
expected for different supervisory and administrative jobs- Р 

(6) The evident practical urge to include oral interven 
makes necessary further improvements in this testing tec 
nique. 


ARMY GENERAL CLASSIFICATION TEST SCORES 
FOR CIVILIAN OCCUPATIONS 


THOMAS W. HARRELL? 


University of Illinois 
and 


MARGARET S. HARRELL 


For more than twenty years research workers have been 
referring to data on intelligence and occupation based on the 
Army Alpha scores and the civilian occupations of enlisted men 
in World War L? The present study deals with similar mate- 
rial It reports the Army General Classification Test (GCT) 
scores of 18,782 white enlisted men of the Army Air Forces Air 
Service Command? distributed according to their previous 
civilian occupation, Means and medians and standard devia- 
tions are reported for the 74 occupations for which there were 
samples of sufficient size to be of significance. 

The data, more applicable to occupations of today than are 
the data presented in earlier studies, supplement present 
knowledge concerning ability levels for occupational groups 
and are of value in educational and vocational counseling of 
civilians and especially, at this time, of discharged soldiers. 

There has been much discussion as to how well the Army 
Alpha sample represented the general population. At the 
Present time it is impossible to decide that issue with respect 
to this sample. It is possible that the averages among the 
Professional occupations are too low since conceivably many 
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the National Acade Sciences, 1921. 
Fryer, Deuda? dccapational-Intelligence Standards.” School and Society, 


XVI (1922), 273-277. . 

Bingham, Walter V. Aptitudes and Aptitude Testing. New York: Harper 
Bros., 1937, pp. 44-59. 

3 Data were furnished by Lt. Col. К. W. Faubion and Lt. Col. J. L. Webster. 
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of the best men in the profession would have been officer mate- 
rial. It is more likely that the averages among the lowest 
scoring occupations are too high since for a while the Army 
Air Forces were receiving a smaller percentage of enlisted men 
with low scores on the GCT than were the Army Ground and 
the Army Service Forces. 

Scores were obtained from Informational Rosters made ир 
from Soldier’s Qualification Cards and other sources. Only 
those cases were included where the roster clearly indicated the 
previous civilian occupation of the soldier by job title. The 
job titles in the following tables are those given and described 
in Army Regulation 615-26, Index and Specifications for 
Civilian and Military Occupational Specialists, or in the U.S. 
Employment Service Dictionary of Occupational Titles. In à 
few instances, however, a general job title has been used In 
this study instead of the more specific breakdowns of the job 
described in A.R. 615-26, i.e., Engineer includes mechanical, 
civil and mining engineers. These general job titles are Engi 
neer, Laboratory Assistant, Inspector, Musician, Foreman, 
Electrician, Assembler and Welder, Manager, and Miscellane- 
ous. Miscellaneous is a general title which includes managers 
of various business establishments, i.e., moving picture theater? 
bowling alleys, etc., but excludes managers of retail stores: 
Manager, Retail Store includes managers of both chain ап 
independent retail stores. Mechanic includes all kinds of me 
chanics except air-plane and automobile mechanics. : 

The desired maximum size of sample for a single occupatio? 
was 500 cases, and when that maximum was reached furth¢ 
tabulation of the occupation was discontinued except in а few 
instances where the range of scores was great. For the major 
ity of the occupations 500 cases were not available. It ko 
originally planned to omit all occupations with less than 4 
cases; some have been included, however, as they inv 
several professional groups in which there is consider? 
interest. Я 

2 In order to provide a rough check on the reliability of aa 
[et "pee ue the scores for each occupation were Е: 
ributions, А and B, of approximately the $% - 


ple 


olv . 
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Mean and Median GCT Standard Scores, Standard Deviations and Range of 


Scores of 18,782 AAF White Enlisted Men by Civilian Occupation 


Occupation N м Median Standard Range 
Accountant .. 12 1281 1281 17 94-157 
Lawyer 91 1276 1268 109 96-157 
Engineer creeren ansas- 39 1266 1258 117 100-151 
Public Relations Man .. 42 1260 1255 114 100-149 
n uditor паа. & 1259 1255 112 98-151 
| БЕШ чуларына ийа 21 1248 1245 158 102-153 
Reporter. 45 1245 1257 117 100-157 
Chief Clerk’ | 165 1242 1245 117 88-153 
eacher .... 256 1228 1237 128 76-155 
[ЕШШ aa n dre oem cU Ws gU cena 13 1220 1217 128 74155 
Stenographer „еее nne tern 147 1210: 1214 125 66-151 
о сарнин melee ` 68 1205 1240 152 76-49 
abulating Machine Operator . 140 120.1 119.8 13.3 80-151 
OOKEHBBE: aera ир-атны. 272. 3200 ШОТ, B 70-157 
GENUS конли MM NNI 42 1190 1207 115 90-137 
Purchasing Agent ..... е 98 187 1192 129 82-133 
anager, Production 34 181 170 160 82-153 
Otoprapher .......... 9; 1176 1198 139 66-147 
lerk, General | 496 1175 179 130 68-155 
Clerk2Typist: „узшд а. ааз нк» 468 1168 1173 120 80-147 
Manager, Miscell .. 35 1160 175 148 60-151 
Installer-Repairman, Tel & Tel -- 96 115.8 1168 131 76-149 
(CEN e BC RAE C NE 111 115.8 1168 119 0-14 
nstrument Repai x 47 115. 5. 11. 2 
EDAM EU 267 1153 165 145 56-151 
Printer, D : n 
) : PD A missin 2y e PUS O2 151 167 143 60-149 
alesman ps 494 1151 1162 157 60-153 
| Uit rg ИЕА 48 1149 a uz КЖ 
| anager, Retail Store .. 420 1140 116: ‘ E 
| aboratory ЖИЕ Mq it Ds 1134 140 14 76-147 
125 116 05 76-143 
1123 131 157 54-147 
118 1130 163 54151 
1113 1134 164 58-155 
1109 128 15 56-147 
пол 1108 161 38-153 
1098 114 167 60-151 
1098 1130 147 68-147 
1093 1105 149 66-47 
1092 1104 163 42-149 
p 109.0 1106 152 64-149 
1085 1094 155 64-147 
1076 1089 158 52-151 
1075 1081 153 62-153 
1071 1088 155 70-133 
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TABLE 1 (Continued) 


Standard 


РЕЗ eT 


Occupation N M Median devidtion Range 
Assembler ease ss алаа 498 106.3 106.6 14.6 48-145 
ТИПШ нола italic 421 163 1083 160 60-155 
Machine Operator .................. 486 104.8 105.7 17.1 42-151 
Auto; Serviceman vias vs ve rer Votis 539 104.2 105.9 16.7 30-141 
КЕКЕНЕ ЕЕРЕЕ 239 1041 1053 151 50-141 
C3bIBSlmaker: «us аеннан ө vs an sanos 48 1035 1047 159 66-127 
Upholsterer ........0.000e0cee cece, 59 1033 1058 145 68-131 
Бег соса 259 1029 1048 171 42-147 
Hlunibéit. uic рна Im e аршы 128 1027 1048 160 56-139 
ШЕЕ рж шск E 98 1022 1050 166 56-137 
Carpenter, Construction ............ 45] 102.1 104.1 19.5 42-147 
ine Witten нака ay deeem ara 72 1019 1052 180 56-139 
Welder ....... £l 493 1008 1036 161 48-47 
Auto Mechanic п 466 10.3 1018 170 48-151 
ЛЕД сой. scams E capte 79 10011 1055 202 48-137 
Сао еп s as акар disse Se occisis 194 1008 10 18.4 46-143 
Tractor Driver „аа $4 1002 loe 191 42-47 
Painter, General .................... 440 98.3 100.1 18.7 38-147 
Crane Hoist Operator .............. 99 97.9 99.1 16.6 58-147 
Cook and Вакег.................... 436 972 995 208 20-47 
ЕБЕ an cà we sis Sa sists: ны Us cip elm 56 97.0 7. 17.7 50-135 
Truck нуе s; 92 978 197 1619 
irr), к UNDO 86 958 977 201 2615 
Dub Оры а M ELE 13 953 981 205 42-10 
Lumberjack 2.1210 59 — 947 — 965 198 46-137 
Pen coste QUU а 700 2 1.8 24-147 
Farmhand сезт: 80 912 910 207 24-159 

E a SE e Amos ea de AA 1 - 

OEE чалы ed EE ate EP E 25 F4 106 46-145 


size. The difference between the means of A and B for each of 
the 48 occupations with 100 or more cases ranged from 0.1 to 
5.1 and averaged 1.7. (The median difference was 1.3.) F°" 
the 26 occupations with less than 100 cases, the difference be 
tween the means of groups A and B ranged from 0.4 to 1 

and averaged 3.7. (The median difference was 3.0.) 

Table 1 gives the mean, median, standard deviation 4” 

range for 74 occupations. The means and medians were ©? 
culated from distributions grouping scores by intervals of two 
The standard deviations were calculated from distributio?? 


grouped to provide from 12 to 18 class intervals. 


s . б * а 
_ Table 2 gives the percentage distribution of each occUP " 
Ae It shows, as have other similar studies, the great ges " 
apping of scores between occupations. Even Teamste^ : 


TABLE 2 
Percentage Distribution of GCT Standard Scores by Civilian Occupation of 18,782 AAF White Enlisted Men 
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* Fewer than 0.5% cases. 
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Where the critical ratio is 3 or more, no entry has been made in the table. 
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Watchmkr 
Airpl. Mech. 
Sales Clk. 
Electrtn. 

R/S Checker 
S, Mtl. Wkr. 
Lineman 
Pipefitter 


Lathe Op. 
Welder 


Auto Mech. 
Molder 
Crane Hst. О) 
Cook-Baker 
Weaver 
Truck Drvr. 


Laborer 
Farmhand 


Assmbler 
Mechanic 
Mach. Oper. 
Auto Serv. 
Riveter 
Cabinetmkr. 
Upholsterer 
Butcher 
Plumber 
Bartender 
Carpenter 
Chauffeur 
Tract Drvr. 
Painter 
Barber 
Lumberj 
Farmer 
Miner 
Teamster 
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occupation with the lowest mean, has 2% of its cases scoring 
as high as the mean of Accountant, the occupation with the 
highest mean. On the other hand the top seven occupations 
have no cases as low as the mean score for Teamster. Evi- 
dently a certain minimum of intelligence is required for any 
One of many occupations and a man must have that much 
intelligence in order to function in that occupation, but a man 
may have high intelligence and be found in a lowly occupation 
because he lacks other qualifications than intelligence. 

Table 3 gives the critical ratios between the means of 
басһ occupation compared with every other occupation. 
Where the critical ratio was 3 or more no entry has been made 
ш the table. The table shows, as does Table 2, both the over- 
lapping and differentiation between occupations. In general, 
the mean of an occupation is not significant within a range of 
approximately = 10 occupations where the occupations have 

een arranged in descending order of means. This range 
Would have been less if all of the occupations had had at least 
Ü cases. It is all the more interesting, consequently, that 
Chemist with 21 cases, Engineer with 39, Public Relations Man 
With 42 and Reporter with 45 cases have means significantly 
igher than those of at least 49 other occupations. 

As is always found in studies of this kind, the professional 
and clerical administrative groups of occupations show the 
ighest average scores. Those occupations with average scores 
of 120 or more are, in the order of their superiority: Account- 
ant, Lawyer, Engineer, Public Relations Man, Auditor, Chem- 
Ist, Reporter, Chief Clerk, Teacher, Draftsman, Stenographer, 
armacist, Tabulating Machine Operator and Bookkeeper. 

Since the GCT is a measure of ability to manipulate words, 
numbers and space relations, it is to be expected that those 
Occupations with the lowest averages on the test are likewise 
the occupations least concerned with words, numbers or space 
relations, Lowest scoring occupations from low to high are 

€amster, Miner, Farmhand, Farmer, Lumberjack, Barber, 
aborer and Truck Driver. 
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MECHANICAL ABILITY, ITS NATURE AND MEA- 
SUREMENT. I. AN ANALYSIS OF THE VARI- 
ABLES EMPLOYED IN THE PRELIMINARY 
MINNESOTA EXPERIMENT 


J. R. WITTENBORN 
Yale University 


A Review of the Minnesota Program 


. THE present study comprises a brief review of the prelim- 
mary Minnesota experiment and a factorial analysis of the 
variables. The purpose of this paper is to apply modern an- 
alytical techniques to the interesting problem of the nature 
of mechanical ability which was begun some two decades ago 
by the authors of the Minnesota experiment (5). 

Although the Minnesota research had as its primary aim 
the discovery and measurement of mechanical ability, the 
ены “mechanical ability” has never been rigidly and unam- 

Iguously defined. Since what is meant by mechanical ability 
as never been perfectly general, procedures for assaying the 
Pi of mechanical ability are held to be appropriate by 
ious investigators to varying degrees. In general, however, 
mechanical ability has been an expression for whatever ability 
ог abilities are required for creditable work with tools and 
machinery, i.e., mechanical work. The exact nature of me- 
chanical ability was not specified by the authors of the 
Mnesota study; it was instead likened by them to intelli- 
gence. It was considered to be a general function, probably 
сошргізіпв constituent parts, the precise identity and impor- 
ance of each varying from authority to authority. 
me question of the organization and composition of me- 
ical ability is, of course, critical for economical testing 
Procedures. Since the publication of the Minnesota studies, 
new analytical techniques have been devised and employed. 
241 
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It is possible that by combining the contributions of various 
investigators a tentative answer to the question of the organi- 
zation of mechanical ability may be offered. The answer 
must be regarded as tentative not only because our researches 
to date are by no means exhaustive but also for a more positive 
reason. The performances which investigators have found 
desirable to scrutinize and to measure are determined by the 
requirements and the characteristics of our culture. А 
mechanical and other practical operations change perhaps old 
requirements become less important and new ones rise to 
importance. Different technical devices will be developed and 
the precise composition of that loosely defined group of abilities 
referred to as “mechanical” must be expected to change- 

Of the several patterns of speculation concerning the com- 
position of abilities, the theory of unique traits has been most 
provocative. This theory has been linked with such names 25 
Hull, Kelley, Thurstone, Thorndike, and Woodrow. It, 1 
general, provides two assumptions: (1) that all variability к 
human behavior may be expressed as a function of a limite 
number of independent, elemental abilities and (2) that thes? 
abilities may be discovered and suitable tests designed. The 
theory had been influencing psychometric research some td 
prior to the publication of the Minnesota Mechanical Ability 
Tests. Its full effect was not felt, however, until the invention 
of a practicable factor analysis by Thurstone. Before t Ы 
development of effective factorial methods, a variety а b 
pedients were employed by those who desired to make explor 
tions in the area of human ability. lly 

In the preliminary experiment, the literature was carefu 
surveyed for the purpose of selecting tests which were Кл 
to possess encouraging validity and reliability. The prac Б 
problems involved in their administration were conside а 
Particular effort was made in the selection to avoid inclu t 
more than one test which appeared to measure the same | 
of mechanical ability. Twenty-six tests were selected - to 
ganized, and in some cases greatly changed with respec 
instructions, conditions of administration, or length. tho 


A А 
the auth i : sized * 
authors claimed that test individuality was emph 


ч 
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the selection, it was possible for them to make a provisional 
classification of the tests under seven general headings. The 
headings indicated the nature of the operation which the tests 
appear to be measuring (5, 43-44). 


I. Standard group intelligence tests 
1. Army Alpha, Form 6 (group paper) 
2. Otis Self-Administering Tests of Mental Ability, 
Higher Examination: Form A (group paper) 


II. Simple motor tests 

Tapping Test A (group paper) 

Tapping Test B (group paper) 

Tapping Test C (individual apparatus) 

Steadiness of Motor Control (individual apparatus) 

Accuracy of Movement or Tracing Paper (group 

paper) 

6. Accuracy of Movement or Tracing Board (individual 
apparatus) 

7. Aiming (individual paper) 

8. Speed of Movement (group paper) 


oe oN 


ПІ. Balancing tests 
1. Body Balancing (individual apparatus) 
2. Stick Balancing (individual apparatus) 


Iv. Complex eye-hand coordination tests 
l. Link's Machine Operator's (individual apparatus) 
2. Card Sorting (individual apparatus) 
3. Card Assembly (individual apparatus) 
4. Packing Blocks (individual apparatus) 

V. Assembly tests involving manipulation and responses to 
spatial relations 
l. Stenquist Assembly (group apparatus) 


2. Paper Form Board (group paper) 
3. Link's Spatial Relations (individual apparatus) 


4. Cube Construction (group apparatus) 


VL Tests of mechanical knowledge 
l. Stenquist Picture Tests I and II (group paper) 
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VII. Miscellaneous tests è 
1. Slow Movement or Motor Inhibition (individual 
paper) 
Digit-Symbol Substitution (group paper) 
Letter Cancellation (group paper) 
Number Cancellation (group paper) 
Rhythm or Perception of Time (group apparatus) 


кз ө 


It will be of interest to compare the authors’ tentative 
classification of the tests with the functional classification dis- 
covered by the present writer’s analysis. Я 

The tests were administered to boys who were enrolled z 
the seventh and eighth grades of a Minneapolis junior hig 
school and whose curriculum included the shop courses. 
number of factors determined the selection of this particular 
group of subjects. Boys were chosen rather than men because 
it was felt that in boys individual differences would be de 
termined less by differences in the amount of mechanicã 
training and more by stable abilities which accrue from us 
eral sources. Boys from the middle class were selected rat bis 
than a more heterogeneous sample. The reason for t " 
selection was that in all probability the boys in the pan 
classes have had a restricted opportunity to acquire ee 
ical abilities and the boys from the low, under-privileged € ye 
have had restrictions in their development, also. Selection, а 
boys who were enrolled іп shop courses was determined by by 
need for a suitable criterion which could not only be ers 
determined but which would also suitably represent the va ica 
operations which are considered to comprise mechan 
ability. The battery of 26 tests required 11 hours of te 
and complete data were collected for 217 boys. 


The Original Analysis of the Variables of the P reliminary 
Experiment . the 

The authors of the Minnesota study were interested 1 100 

organization of mechanical ability because of its нир e val 

for this very practical question: Is it possible for an 106! n 

to be distinctly gifted in one line of mechanical work here d 

possess poor or mediocre capacity in others, Le. 18 
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mechanical ability or group of abilities? In their attempt to 
answer this question, they intercorrelated all of the variables 
employed in the preliminary experiment and then subjected 
these intercorrelations to analytical procedures. Since Spear- 
man’s theory of general ability, particularly with respect to 
intellectual functions, was one of the most arresting theories 
of the day, the authors concerned themselves principally with 
the task of finding evidence of a general mechanical ability. 

In order to fulfill Spearman’s demand that only dissimilar 
tests be subjected to scrutiny when a general factor is sought, 
the authors carefully examined their own tests and selected a 
group of 15 tests which were dissimilar to each other and were 
Not correlated with each other to an exceptional degree. In 
light of current factor theory, the implications of this selective 
Procedure are obvious. Similar tests are likely to be those 
highly intercorrelated and if several highly intercorrelated 
tests are encountered in a group of tests, we are certain to find 
а cluster or a factor. If such a cluster were found, it would 
not be possible to account for all of the inter-test correlations 
1n terms of one general factor. 

The intercorrelations of the selected 15 variables were ar- 
ranged in the usual tabular form, and the inter-columnar cor- 
relations for this table varied from minus .68 to plus .81. It 
is obviously impossible to find any hierarchical order in such 
а table. A similar lack of hierarchical arrangement was en- 
countered when the variables were corrected for attenuation. 

he results of this procedure combined with the results of 
Certain preliminary procedures forced the authors to the con- 
clusion that their data offered no evidence for a general factor 
™ mechanical ability. . 

The authors then sought evidence for the existence of group 
factors. On a trial-and-error basis, attempts were made to 
Construct hierarchies by arbitrarily selecting certain tests for 
Srouping. This procedure was adopted because group factors 
are general within their own restricted field, that is, variables 
which define a group factor may be arranged in hierarchical 
order. The authors assumed that if several well-defined hier- 
archies were disturbed when tests from any other hierarchy 
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were included, the evidence would be strongly in favor of the 
existence of group factors, that is of mechanical abilities rather 
than a general mechanical ability. The writers reported that 
they found seven perfect hierarchies of four tests each. Two 
of these hierarchies were found to be mutually exclusive with 
respect to their component tests. The other five hierarchies 
drew upon the same tests. The writers do not indicate what 
variables contribute to all of the hierarchies. Since such hier- 


archies can be arbitrarily constructed without one's knowing . 


to what degree they overlap, the analysis of the organization 
of mechanical ability provided by the Minnesota study leaves 
much to be desired. From their presentation it may be 10- 
ferred that if a general mechanical ability factor is present; it 
is certainly not sufficient to account for all of the inter-test 
correlations. It may be further inferred that group factors 
do exist, but the degree of independence which they posses 
and their exact nature is in no way manifest. 


A Factor Analysis of the Variables in the Preliminary 
Experiment? 3 
The Minnesota investigators had employed numerous Ms 
ables which are representative of if not identical with тапу г: 
use today. Their population was quite large and in the! 
attempts at the analysis of the organization of mec $ 
ability they determined the intercorrelations among all of t M 
variables. An analysis of the Minnesota data using M 
which were not then available is in order in two respects: E 
to round out the classic Minnesota investigation and СА. 
shed light on the current problems of measurement of mec 
ical ability. As a consequence the intercorrelations 0 2 
Minnesota variables given in Table 1 were submitted hich 
centroid analysis; seven factors were extracted, six of W for 
are given in Table 2. The centroid analysis was made, o 
twenty-seven variables. The tests were the original lis 


me 
de x А str" 
twenty-six variables with the exception of the cube € 
tion test, which was not intercorrelated by the Min scary 
— M in: 


У yi E elim үа 
+ This analysis is concerned only with the variables employed in the Pines?" 

experiment In a forthcoming publication additional analyses of the 

data will be reported. 
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authors with the other variables. This left twenty-five tests. 
The Stenquist Picture Test, however, really comprised two 
parts, I and П. Each was scored separately, correlated with 
the other variables separately, and the two parts are treated 
as two variables in the present study. Age was also included 
as a variable and thus the total list of twenty-seven variables 
which the present writer has analyzed is comprised. 


TABLE 2 
Centroid Matrix 

A B c D E F p 
1 00 25 -.19 =37 -.26 -.08 2 
2 29 06 26 14 03 -.20 76 
3 34 -46 35 25 43 25 28 
4 31 09 23 26 =:22 -.09 46 
5 50 18 –.24 31 -.10 13 63 
6 .53 12 =22 49 -.10 20 41 
7 48 26 -.09 22 22 09 64 
8 44 16 – 35 -.10 48 —.25 41 
9 50 27 -.06 a7 -.15 18 45 
10 52 -.30 =28 08 -.02 07 6% 
11 40 28 —.38 -.13 33 -37 62 
12 35 -46 31 18 29 27 45 
13 49 - 05 -20 18 10 15 $0 
14 48 -47 „Д7 -04 S» 08 10 
15 14 06 15 20 10 -.09 13 
16 23 -.11 =22 -.05 10 -.08 37 
17 44 29 10 07 03 11 22 
18 27 08 123 108 —.04 -26 48 
19 43 -.36 - 25 =28 = 28 -04 44 
20 30 - 42 -.18 =29 - 20 -45 71 
21 57 nt =a aD -.06 05 20 
22 20 29 15 13 221 -.03 37 
23 33 32 17 EN 14 16 55 
24 34 -.38 08 -4 -.16 34 50 
25 44 29 17 - 35 -.18 23 30 
26 30 ll 06 17 E - 32 17 

27 19 - 04 23 ‘05 all -27 


Р ding 
The centroid factors were orthogonally rotated accor 


to the well-known principles of maximizing the nu nes: 
zero loadings and minimizing the number of negative ? nd 
After thirty-six sets of rotations had been made it was ae 
that further rotations would contribute little to the simple 
of the patterns revealed. Each of the six orthogonal ма 

is well defined, and for the first five factors the loading’ in 
exceptionally high. The rotated factor matrix is Ps 4. 
Table 3 and the transformation matrix is given in ^? 


mber 
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TABLE 4 


Transformation Matrix 


A B G D E E 
I 35 -64 =й] - 45 -31 OF 
Hi 45 49 39 -.54 -20 24 
ш 35 -.48 51 24 А1 = 
IV ‘62 25 aJ ‘61 -24 0 
у 35 A5 2:97 a23 74 - 42 
VI = 31 17 -47 21 29 78 


In order to clarify the implication of these factors, each will be 
discussed with respect to its component variables and with 
respect to the degree to which component variables are satu- 
rated with other factors. 

Since the factors are discussed variable by variable, the 
data will be presented in the form of factorial equations. 


Factor I—Spatial Visualization 


y? 

I ш Mm iav v b: 35 

10. Link’s Spatial Relation ...... 21 00 05 16 0 a "51 
14. Paper Form Board .......... E .00 .08 .06 .00 01 151 
19. Stenquist Assembly ........ 44 01 0 02 0l n 155 
20. Stenquist Picture I ........ 40 .00 .00 .00 .01 01 29 
21. Stenquist Picture II ........ 5 .00 11 02 .02 E f 


The Spatial Relations Test of Link is the original form i 
the currently popular Minnesota Spatial Relations Test- dified 
Paper Form Board Test from the Army Alpha is now mo The 
and known as the Minnesota Paper Form Board Test. dified 
Stenquist Assembly Test has been lengthened and то 7 
and appears as the Minnesota Mechanical Assembly ela- 
Twenty-one per cent of the total variance of the Spatial er 
tions Test is due to the spatial factor. Another sitem 
cent of its variance is due to factor IV, which is ident! if 
a dexterity factor. Users of the Minnesota Spatial Rel у 
Test have long believed that this test calls for spatial А ilit 
and for a certain amount of dexterity or manipulative 5 cent 
as well. The Paper Form Board Test has thirty-five Рё 
of its variance attributable to the Spatial factor; abou 

2 Factorial equations are more revealing than simple factor loading 
Foe р FON much of the total variance of a test is due cas 

e how much of the variance is not due to . the 


n . "n H e. 
18. How much is unique (U?) to the test in the present test an ared 
in the factorial equations are equal to the respective factor loadings 


a 


4 
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per cent is due to factor III, which in this study is called the 
Scholastic ability factor. The Assembly Test is largely a 
measure of spatial ability. It is very interesting to observe 
that it calls for the dexterity factor, IV, to a negligible degree. 
Contrary to the implications of its name, the Assembly Test 
calls for the ability to visualize the relations of parts rather 
than the ability to excel in the manual process of assembling 
parts. 

Some explanation should be offered for the difference 
between the saturations of tests 20 and 21, both of which are 
Stenquist picture tests and contribute greatly to the spatial 
factor. Tt is observed that test 21 owes approximately 10 per 
Cent of its total variance to the scholastic ability factor. A 
Possible explanation of this difference between the two picture 
assembly tests is that test 21 was given under marked restric- 
tion of time. The time for each exercise in the test is reduced 
twenty per cent. The meaning of time restriction for this 
Saturation of test 21 is a question that will be discussed further 
Under factor III. 

Factor II is determined by tests which appear to call for 
а high degree of speed in certain stereotyped ballistic move- 
ments of the wrist and forearm. 


Factor II—Stereotyped Movement 
I TI^ enr. wy Vi VI U2 


(9. Lines M 30 00 .00 59 
ach. n) 10 .01 : A à 2 

2 Speed of dum PE: Q0 30 (Q0 о и Q0 6 

DIN Lopine А)... „шише о з 01 0 45 O0 6 

BC ОЕ ОЕ О-у ш o X0 2 43 

ШАРЫН C: шогыры XO 4б 00 02: 00. оода 


The Speed of Movement Test calls for as many vertical 
Marks as possible within a time limit. Variables 23, 24, and 

Were all tapping tests which were conducted in different 
Ways. In test 23, the individual is to make one dot in each 
9f the large nütibér of tiny squares. In test 24 he is to make 
as many dots as possible in each of the large squares outlined - 
оп his test paper, fifteen seconds allowed per square. In test 
5, the individual is equipped with a metal plate and a stylus 
and his rate of tapping is determined. In all four of these 
tests the individual must make very rapid movements of the 
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wrist and the forearm. It is observed that test 9 had ten 
per cent of its total variance attributable to this factor. The 
machine operator’s test, 9, involves dropping balls into а 
funnel at intervals which permit the ball to drop through a 
momentarily appearing aperture in a platform beneath the 
funnel. This performance calls for a vertical dropping move- 
ment of the hand which is not unlike the vertical movements 
called for in the various tapping tests. 

Factor III is called the Scholastic Ability Factor for the 
simple reason that two well-known tests of scholastic ability 
comprise it. 

Factor III—Scholastic Ability 


I IH I IV V VI 
a. 
4. 


0 ài о o M 


These tests are the Army Alpha and the Otis. The € 
are not intelligence ratios but are simply raw ee 296. 
would be expected, this factor is negatively correlated wit’ к 
It will be remembered that the subjects for the prelio z 
experiment were picked from the seventh- and eighth e tas 
classes. Ordinarily the older children in a public school € tly 
are the children who have been retained, and most freut 
this retention has been due to dullness on the part of the p it 
If this general observation may be validly applied a age: 
accounts for the negative relation between this factor "picture 
It is further observed that variable 21, the Stenquist actor 
Test П, has a probably significant relationship with p ctor } 
The next variable most highly correlated with this ta [phe 
variable 14, the Paper Form Board Test. The Army 
the Otis, the Paper Form Board, and the Stenquist assifie 
Test II are all pencil-and-paper tests which may be © s thes? 
as mental tests and all four are highly speeded. Регһар ive 
common characteristics can account in part for the P 
intercorrelation. call i 

Factor IV is determined by tests which appear t? 

a type of manual dexterity. 


12. 
(21. 
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Factor IV—HManual Dexterity 
I п ш IV у VI 2 


© а 0 о п 0 o т 
e 00 1 00 43 пт ә 5 

б: 00 00 03 9 00 O1 36 

f 02 103 о 23 10 п 58 

im 00 10 010 30 0 00 59 
its п о 05 во п 535 
à Q3 00 в 2 и ә 86 


Variables 5 and 6 are the best measures of this factor and 
they are the well-known Card Sorting and Card Assembly tests. 
The next best test appears to be test 9, which is the Machine 
Operator’s Test described in the discussion of Factor II. Tests 
Which have correlations of approximately .30 or greater with 
this factor tend to contribute to its meaningfulness; variables 
4 and 7 аге possible exceptions. There is no apparent reason 
Why variable 4 should be expected to correlate with the factor 
of manual dexterity, and it is somewhat surprising although 
not wholly unreasonable to find a test of digit-symbol-sub- 
Stitution highly correlated with the dexterity factor. The 
exact nature of the manual dexterity factor is not perfectly 
clear. To what degree excellence of performance on tests of 
this factor is due to a type of acuity of perception and recog- 
Nition has not been determined. This question will be dis- 
cussed further with presentation of Factor V. Test 10, the 

Patial Relations Test, which owes sixteen per cent of its vari- 
ance to this factor, obviously requires manipulative skill and 
Would be expected to draw upon some sort of manual dexterity 
to a significant degree. 

Factor V is identified by tests which have been term 
Perceptual tests and have been found by previous investigation 


t ^ 
© determine a perceptual factor (9). 


termed 


Factor V—Perceptual Factor 
I п II IV у УІ U2 


7 

: 0; 03 23 10 01 58 

ll. 2 0 0 06 57 0 36 
i à o .0 07 а Q0  .36 


_ The two cancellation tests, no. 8 and 11, have remarkably 
igh saturations with this factor. Test no. 7, the Digit- 
gta Substitution Test, has unexpectedly low saturation. 
€ reason for this test's having a high saturation with the 
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dexterity factor rather than with the perceptual factor may 
actually be due to an undetected clerical or computational 
error. On the other hand, it is conceivable that the relations 
observed here are valid relations and perhaps some unrecog- 
nized aspect of the digit-symbol performance could, if under- 
stood, give insight into the nature of manual dexterity- It 
should be observed in passing that tests commonly employed 
in the measurement of perceptual ability, including cancella- 
tion tests such as appear in this battery, call for speed of 
simple, routine perception and the emphasis is not upon acuity 
of differential recognition and discrimination. Perhaps manu? 
dexterity is a complex ability which does call for a differenti 
recognition and discrimination which is not purely or primarily 
of a routine nature. The hypothesis that tests for manu? 
dexterity draw upon two distinct abilities, a manipulativ® 
ability and a recognition ability, may be tested by determining 
the factorial composition of manipulative tests when give? 2ч 
a darkened room or under conditions to eliminate visual dis 
crimination and comparing the results with the factorial comi 
position of these tests when they are administered under Oe 
nary conditions. Factor analysis is not necessary to test t ns 
hypothesis, however, and other, simpler designs would " 
doubtedly be more economical. 


H р jnter- 
Factor VI has low saturations. The relatively low 4; 


à ; ч eadi- 
correlations among the tests which contribute to е 
ness factor are in part due to their unreliability. The, ics 

teristie”? 


with the highest loadings have certain common charac 
and as a consequence the factor of steadiness has 
hypothesized. 

Factor VI—Steadiness 1° 


у : 
їшї ш у a «О 


ol 02 oa о | УД; 
: we QU 02 * Or i Шш 1 b 
18. Steadiness, No. contacts .... 01  .02 01 02 l 7j 83 
26. Tracing Board, No. errors .. 00 00 00 08  .00 ; 
27. , Tracing Paper, No. errors .. .00 01 00] 00 .00 self 


Body Balancing calls for the individual's balancing I! cube 
on the ball of one foot while standing on a three-in* dines? 
of wood. Buxton (2) reported a factor of manual ec 
based upon two tests, one of which was a thrusting c 


< 
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calls for an operation similar to the nine holes steadiness test. 
In a later paper, Seashore et al. (7) reported a steadiness factor 
which was based upon measurements of postural sway. The 
steadiness factor which is reported in this study is dependent 
upon a measure of postural steadiness, body balancing, as well 
as measures of manual steadiness. These data in combination 
with the data provided by other observers (4, 6, 8) suggest 
that there is one general steadiness factor (probably due to 
basically physiological individual differences conceivably in 
Proprioceptive sensitivity or muscular tonus). The possi- 
bility that there exists only one general factor of steadiness 
has quite useful implications and the hypothesis may readily 
be tested. 

The conclusion is: apparent from this analysis that no 
common factor could account for the intercorrelations among 
the various tests. On the contrary it is apparent that the 
Inter-test correlations may be accounted for in terms of six 
Independent and meaningful factors. Obviously such factors 
Cannot be considered immutable or part of the universe, but 
among tests which the authors of the Minnesota study (as 
Well as current investigators) consider to be important and 
representative, meaningful, well defined, independent, func- 
Чопа] groupings have been demonstrated. 


Other Analyses of M echanical Ability 

There are few factorial studies relating to mechanical 
ability reported in the literature; the most satisfactory one is 
. arrell's, which he succinctly summarizes as follows: “The 
Mtercorrelations of thirty-seven variables, including the Min- 
nesota battery of ‘mechanical ability’ tests, the seven 
acQuarrie tests of ‘mechanical ability; O’Connor's Wiggly 
locks, and the Stenquist picture-matching test, Were analyzed 
Y Thurstone's centroid method. Five factors, Perceptual, 
erbal, Youth, Manual Agility, and Spatial, were taken out. 
actors prominent in so-called mechanical ability tests are 
the Spatial and Perceptual ones with MacQuarrie’s dotting test 
e nificantly high in the Manual Agility factor. Each of the 
actors can be measured with group pencil-and-paper tests 
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(3). Harrell’s Factor IV, agility, could readily be identified 
with the present writer’s manual dexterity factor, IV, were it 
not for the participation of the dotting test in Harrell’s agility 
factor. Since there were no additional dotting tests in his 
battery or other tests of the ballistic type, we cannot be con- 
fident that the loading of the agility factor with this particular 
test has general significance. It may be suspected that this 
test is highly related with a “ballistic” or stereotyped move 
ment factor; high correlations between MacQuarrie’s dotting 
and tapping tests have been reported (1). 

In “The Application of Multiple Factorial Methods to the 
Study of Motor Abilities” (2) Buxton found several factors 
which are in essential agreement with the results of the present 
study. His Factor I is a dexterity factor which appears to 9 
identical with the dexterity factor described іп the present 
study. His Factor III, a steadiness factor, is determine by 
manual tests only. Buxton's Factor VI is found in neither t Р 
present study nor in Harrell’s study. It appears to C? em 
general coordination of the muscles of the forearm, upper am 
and shoulder girdle. qo by 

A “Multiple Factorial Analysis of Fine Motor Skills dh 
Seashore, Buxton, and McCollom has been reported (7). 
results of this study have uncertain value for the broader Ў, 
lem of mechanical ability for several reasons. Тһе tests + 
ployed are of a rather specific nature and may not in aren 
be considered representative of tests for mechanical pm 
The analysis was based upon a sample of 50 men, а hat 
sample for factor analysis. Moreover, it appears likely 
the authors might have extracted additional factors; from 
examination of the factor matrix it does not appear t As 
zero-order correlations may be satisfactorily reproduc 
the writers themselves note, some of the factor load! 
inconsistent with the identifications of the tests алс dy t° 
Although the study was not intended to contribute direc 
the problem of the nature of mechanical ability, it does ан ві, 
the implications of the present study by suggesting © и гур° 
ence of a postural steadiness factor and a factor of 500160 I i” 
movements of the arm such as are called for by Factor 
the present study. 


prob- 


MECHANICAL ABILITY, ITS NATURE AND MEASUREMENT 257 


The literature offers several interesting studies of the or- 
ganization of strength, agility, and motor fitness. In a com- 
prehensive approach to the study of the nature of mechanical 
ability, cognizance must be given to such possible components. 

his aspect of mechanical ability will be treated in a forth- 
coming study. 
Discussion 


_ The occurrence of an orthogonal factor structure is of some 
interest because both Buxton and Harrell have found an 
oblique structure to be most appropriate in analyzing their 
Variables. The orthogonal feature of the present analysis may 
be in part due to the age of the subjects. They are seventh- 
and eighth-grade boys; the subjects for the other analysis of 
Buxton and Harrell had been adults. Babcock and Emerson, 
n а study of MacQuarrie’s test for mechanical ability, have 
ound intercorrelations among the subtests to increase in 
Magnitude with age of subject (1). This interesting tendency 
i5 quite the reverse of that usually observed for the so-called 
Mental tests. The tendency for mental abilities as defined by 
factor analysis to become increasingly independent of each 
other with increasing age and the apparent tendency of me- 
chanical abilities to become more highly related with each other 
With increasing age requires explanation, and it is possible 
that the final answer to this paradox may yield some insight 
into the nature of the respective classes of ability. Since all 
the mechanical tests employed in this study (and in other 
Studies too) are not equally valid, it cannot be argued that 
Excellence in practical mechanical work requires a high degree 
of all of these abilities and that therefore а type of automatic 
Selection occurs which eventually results in those who have a 
igh degree of all abilities being active in mechanical opera- 
tions and those deficient in any опе being equally discouraged 
Mall. It is not likely that any selective situation such as this 
©ап be invoked to explain this paradoxical trend. A slightly 
modified hypothesis may be offered, however; namely, that a 
nigh order of spatial ability is necessary for personally gratify- 
mg performance in any mechanical operation and that persons 


Who are deficient in spatial ability may in general tend to 
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avoid mechanical work or participate with little zeal and as a 
consequence neglect the development of any purely manual 
facilities with which they might have started life. The ap- 
parent anachronism in the development of mechanical ability 
is provocative of speculation. Although the trend may not be 
verifiable its implications are important enough to warrant 
scrutiny of any suitable genetic records which may exist. 

It is of interest to compare the classification of the tests 
arbitrarily imposed by the authors at the beginning of the 
Minnesota investigation and the classification revealed by the 
empirical evidence offered by the intercorrelations and by 
subsequent factor analysis. The first arbitrary classification 
was Intelligence and this classification is validated by the 
results of the statistical analysis. The second classification, 
Simple Motor Tests, actually includes measures for two factors 
defined by the present factor analysis, i.e., Factors I and П, 
stereotyped movement and steadiness. The third group 
comprises the two balancing tests and is also included in the 
steadiness factor. The fourth group, Complex Eye-Han 


Coordination Tests, comprises tests which determine Factor 
IV, identified in the present study as manual dexterity- e 
d o 


participation of the Digit-Symbol Substitution Test an 
other measures in this factor suggests that the designation eye 
hand coordination for this class of ability may be most appro 
priate. As previously mentioned, however, further research 
is required before the exact nature of manual dexterity ©” 
be determined. It may well be that manual dexterity is 4” 
operational unit which when analyzed for logical content € 
demand a type of manipulative ability and visual discrim 
nation. It is equally possible, however, that manual dexterity 
or eye-hand coordination may be fractionated into tw rela- 
tively independent components, visual discrimination and ? 
manipulative ability. Group V, designated as Assembly Т 
Involving Manipulation and Response to Spatial Relation” 
appears to be appropriately identified insofar as our analy*" 
has revealed that these tests, though primarily dependent ш 
the spatial visualization ability, draw in varying degrees ир 

manual dexterity. This class probably was regarded “i 
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мш кр, however; ie. it is not apparent that the 
о bus - foresaw that such tests as the Spatial 
енне бо (а the Assembly Test would be primarily 
De oema yt mas factors as the Paper Form Board Test. 
ee m that spatial ability is a primary component 
MN mg tests as the Stenquist Assembly as well as the 
The Sten n po Test was probably first made by Harrell. 
Penn q sud icture Tests, set aside in a separate class as 
E omen da anical knowledge, are found actually to con- 
comede e spatial factor. Apparently the perceptual factor 

y unanticipated by the Minnesota authors; the 


cance : Ў = 
llation tests are classified under miscellaneous. 


Summary 


Ra present study comprises a review of the preliminary х 
he a experiment and a factorial analysis of the variables. 
of ү. ysis reveals that interrelationships of the variables are 
depend a nature as to yield upon analysis six meaningful in- 
серо ent factors. For the most part the loadings are ex- 
tion d high and the results of the analysis are excep- 
ally definitive. The factors are: 
T—Spatial Visualization 
II—Stereoty ped Movement 
III—Scholastic Ability 
IV—Manual Dexterity 
V— Perceptual— Speed 
V]—Steadiness 
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SUGGESTIONS FOR THE CONSTRUCTION OF 
MULTIPLE-CHOICE TEST ITEMS 


CHARLES I. MOSIER, M. CLAIRE MYERS, лхо HELEN G. PRICE 
State Technical Advisory Service, Social Security Board, Washington, D. C. 


In wnrrING items in a particular area it is possible, but not 
Very profitable, to use the inspiration technique. One reads 
until he is inspired to write an item, jots it down, and then reads 
Some more. Those concepts that fall readily into item form 
Вес tested over and over again; those more difficult to test go 
untested. This procedure is likely to result, among other 
things, in very spotty coverage of the subject-matter area. In 
Writing items, as in other activities, planning is essential. 

The present paper presents three work tools for the con- 
Struction of multiple-choice items: definition of the subject- 
Matter area to be covered and systematic sampling of it; a 
check list of the kinds of questions which can be asked; and a 
Summary of criteria for multiple-choice items. 

Although these materials were developed for use in ex- 
amining for public personnel selection, they are applicable, 
With minor changes, in the other fields of test construction. — 

To cover a subject-matter area adequately, whether in 
educational testing, merit system examining, or aptitude test- 
106, the first step should be a definition of the area or areas 
to be tested. Even in a limited field, all possible questions 
Cannot be asked; it is necessary to resort to а sampling of the 
field. If, however, the individual’s performance on the sample 
18 to represent his performance in the entire field, it is a truism 
that the sample must represent the field; the sample to be rep- 
resentative cannot be left to chance, but should be planned. 

he definition of the field states the limits which must be 

reached; it does not prohibit going beyond those limits. If 

items are written outside the boundary of the definition, they 
261 
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can usually still be used; if no items are written on an area 
within the field, that part of the area has not been sampled. 
A picture may help to visualize this. 

The entire circle represents the particular subject-matter 
area, e.g., knowledge of elementary and intermediate statistics. 
Each subdivision represents a set of related concepts within 
the broader field. For example, Z might be the concepts re- 
lating to frequency distributions; 2 those relating to central 
tendency; 3 those relating to dispersion, etc. Unless the total 
area is defined by correctly drawing the total circle, e.g., closing 
the circle in the lower right quadrant, the existence of the sixth 
subdivision escapes our attention. 


ох ) 


It is essential, if one is to test adequately for а knowledE" 
of the principles and techniques in statistics, that he be Sn 
pared to include test items in each of the areas. It is not a: ne 
sary to include one from every area in every examination? one 
should not, however, include three from section 1 and rept 
from section 6 unless he is prepared to say that the cori 
in J are very important and those in 6 are inappropri? 
the purposes of the particular examination. denti- 

After the area has been outlined, the next step is the "соп 
plished by listing the important concepts, topics, princiP nich 
subdivisions of the field or of the various types of skr - ation’ 
contribute to the total. This list represents the speci®c 


v 
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usps dubbi It should not, generally, be a list of the 
у ris but of the sub-areas to be covered, with 
Eon 6 sampled by several items. Preparing such a 
Es foni ы X an authoritative outline already set up, or 
ү ана = iarity with the field. The chapter headings 
Eom i e paragrap headings within a chapter constitute 
nmn! de a listing. Once made the list is important: 
à zaide "ir ex to the resulting assembly of items; second, as 
laborare A ern item construction; and third, as a more 
Pr iidem nition of the types of items contained in the 
Жусып ter area. Table 1, showing an outline of statistics, 
ed as an illustration, not necessarily as a model. 


TABLE 1 
Я Outline of Statistics 
„© 
үш р 5. Dispersion 
iterpretations General 
Life Tables Sigma 


‘ з 
‘Severity Rate” 
Sanble computations 
ees and bibliographies 
z ular presentation of results 
fime and symbols 
anning surveys 
уы mpound interest formula 
Chee КОНЕ 
lass intervals and limi 
imodality с 
ymmetry 
kewness 
Frequenc 
ncy polygo 
Kurtosis Ж 
ercentiles 
S z 
aoa curve properties 
3. Chan (m expansion 
ts, Graphs 
Orie phs and Index Numbers 
E average 
аще equation 
пег SE Pr 
Werbolation and extrapolation 
nae ams and circle diagrams, 
chang Lorenz, ratio, and time 
SP 
арыр computation 
ae ех numbers 
entral Tendenc 
Mean 4 
Median 
ode 
armonic Mean 
eometric Mean 


Average deviation 
Quartile deviation 
6. Correlation 
Interpretation and use 
Scatter diagram 
Pearson 
Other coefficients 
Regression lines 
Partial and multiple 
Coefficient of alienation 
Standard error of estimate 
7. Non-Linear Regression 
Trend lines 
Population curves 
Eta and Blakeman’s test 
8. Sampling 
Methods: random, weighted 
Sampling errors 
Interpretation, S.E., and С.К. 
Combination of samples 
Probability theory 
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Once the list of concepts, principles, skills or topics to be 
tested has been completed, appropriate sources or sections of 
the source should be used in constructing the items. | 

It is useful to distinguish between the concept, or the skill, 
being tested and the kind of question asked to test that con- 
cept. A further distinction which should be made is between 
the kind of question asked (e.g., what, why, who?) and the 
form of the test item used to ask the question (e.g., true-false, 
multiple-choice, completion). We are concerned here pe 
marily with multiple-choice items and the form of the tert en 
will not be considered further. In Table 2 is a check pete 
some of the kinds of questions which may be asked togethe 
with illustrative examples based on the concept of centra 
tendency. 

TABLE 2 
Types of Questions 


1. Definition 
. What means the same as... . 
What conclusion can be drawn from... . . aa? . 
с. Which of the following statements expresses this concept in different ter! and divid- 
Example: The value which is determined by adding all of the scores 
ing by the number of cases is known in statistics as the: 
(1) arithmetic mean; 
(2) median; 
(3) mode; 
(4) harmonic mean; 
(5) average deviation. 


on 


2. Purpose 

- What purpose is served by... . 

- What principle is exemplified Ьу... . 

Why is this done... . 

‚ What is the most important reason for . . . . "€ 
Example: The mean is obtained for the purpose of providing: 
(1) a single number to represent a whole series of numbers; 
(2) the central point in a series; 

(3) a measure of group variability; | 
(4) an indication of the most frequent response given; 
(5) an estimate of the relationship between two variables. 


ао cp 


3. Cause 
- What is the cause оѓ... . 
. Under which of the following conditions is this true . . . - 
Example: From which of the following measures of centra 
sum of the deviations equal zero? 
(1) the mean; 
(2) the mode; 
(3) the median; 
(4) an arbitrary origin; 
(5) any measure of central tendency. 


will tb? 


op 


1 tendency 


ү 
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4. Effect 
а. What is the effect of . . . . 
b. If this is done, what will happen? 
C. Which of the following should be done (to achieve a given purpose)? 
xample: The arithmetic mean of 55 cases is 83.0. If 3 of the cases, with values 
of 82, 115, and 130 are deleted from the data, the mean of the remaining 52 cases 


will be: (1) 81.50; (2) 77.05; (3) 83.00; (4) 84.50; (5) 94.08. 


5. Association 


Ка uns to occur in connection (temporal, causal or concomitant associa- 
n) with... 

Example: If the distribution of scores is skewed positively, the mean will be: 

(1) lower than the median; 

(2) the same as the median; 

(3) higher than the median; 

(4) relatively unaffected; 

(5) the same as the mode. 


6. Recognition of Error 


Which of the following constitutes an error (with respect to a given situation)? 
= xample: The mean should not be used as the measure of central tendency 
en: 
(1) the distribution of scores is significantly skewed; 
there are a large number of cases; 
а non-technical report is to be prepared; 
(4) the data are continuous; 
(5) other statistical formulae are to be computed. 


7. Identification of Error 


È What kind of error is this? 
ey at is the name of this error? 


at recognized principle is violated? 

“xample: In Compute the mean of a distribution from grouped des ths sina 
of the deviations above and below the arbitrary origin were бш о! y Ге 

; respectively. The final value for the mean was in error. the following 


Possibilities, that one which is most likely to have caused the error is that the 


computor: И 
1) failed to note the correct sign in adding the mean of the deviations to the 
assumed origin; 
(2) used an assumed mean higher than the true mean; 
(3) omitted some of the cases in tabulating the data; 
(4) divided by the wrong number of cases; 
(5) multiplied by the wrong class interval value. 


8. Evaluation 

en purpose) and for what reason? - 
ll, e.g., less than 20, and the magni- 
an assumed mean in the computa- 


1 values; . 

(2) likely to distort the value obtained by the introduction of a constant 

(3) error; 1 
more accurate than the use of actual values; . 

(4) neither better nor worse than computation by other methods; 

(5) applicable only if the distribution is reasonably symmetrical. 


9. Difference 


What j А 
hat is the important difference between . . - - 
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ор» 


Example: Of the following statements, that one which characterizes the essential 
difference between the mean and the median as measures of the central tendency 
of a distribution is that: : 

(1) the magnitude of each score does not contribute proportionately 

computation of the median but does for the mean; 

(2) the median is a point whereas the mean is a distance; 

(3) the mean is less affected by extreme values; 

(4) the median is easier to compute; 

(5) the median is more generally used. 


to the 


10. Similarity 


What is the important similarity between... . 
Example: The mean and median are the same in that they are both m 
(1) central tendency; 
(2) distance; 
(3) position; 
(4) variation; 
(5) relationship. 


easures of: 


11. Arrangement ich 


d h 

In the proper order, (to achieve a given purpose or to follow a given rule) wt 
of the following comes first (or last or follows a given item) — , jntervals 

Example: In computing the mean for data already grouped in class 
the most efficient first step is to: . А 

(1) determine the arbitrary origin and enter the deviation values; 

(2) find the midpoints of the class intervals; on he inte 

(3) multiply the frequency in each interval by the midpoint of the 

(4) add the column of scores; 

(5) find the reciprocal of the total number of cases. 


rval; 


12. Incomplete Arrangement 
In the proper order, which of the following should be inserted 


the series? . 
Example: In deriving the formula for computing the mean from £ 


using an arbitrary origin the following steps were taken: 


e 
here to complet 


rouped data 


а. x= XA, 

b. EX ziEX' + NA; 
ZX as DA 

в. тг At 


The step which is implied between steps (a) and (b) is: 
(1) solving (a) for X; 
(2) summing (a) over the N cases; 
(3) multiplying by i; 
(4) adding A to both terms of (a); ` 
(5) dividing by N. 


13. Common Principle 


:nciple: 
All of the following items except one are related by a common ргпор 
What is the principle? 


. Which item does not belong? edia® 


Which of the following items should be substituted? " ; an, ЛЕ rics: 

Example: All except one of the following items (arithmetic in statis ded 
mode, and quartile) are measures of central tendency; of the fo! roperly int 
Her one which could be substituted in the series for the item ітр 
is the: 

(1) harmonic mean for quartile; 

(2) average deviation for mode; 

(3) range for quartile; 


f 


s= 
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(4) standard deviation for quartile; 
(5) 50th percentile for median. 


14. Controversial Subjects 


Although not every one agrees on the desirability of ———, those who support 
its desirability do so primarily for the reason that: 

Example: Although not every one agrees that the mean is the best measure of 
central tendency, those who advocate its general use base their recommendation 
on the fact that the mean: ы 

(1) has the smallest sampling error; 

(2) is easiest to compute; 

(3) is most readily understood; 

(4) is the most typical score; 

(5) is not affected by extreme values. 


Frequently a concept can be tested by a variety of ques- 
tions; for some concepts only one kind of question is appro- 
Priate; for still others, several are applicable, but one or two 
are clearly most appropriate. To secure adequate sampling 
In the construction of items on any concept, the concept should 
be checked against the list and as many items written as seem 
appropriate, each asking a different kind of question. By 
writing items which test knowledge of a concept through sev- 
eral kinds of questions, it is frequently possible to sample the 
area at several levels of difficulty, ranging from the simplest 
to the most difficult. , 

The construction of a number of different items on each 
Concept included in the outline of the subject-matter field pro- 
Vides a more effective means of meeting the objective of those 
Who feel that internal weighting is essential to give the appro- 
Priate emphasis to the more important concepts. The pro- 
Ponents of internal weighting argue that since certain concepts 
аге more important, extra credit should be given for the ques- 
tons on those concepts. If the desirability of such extra 
Credit is granted, the weighting is more easily and more reliably 
accomplished by including extra questions than by doubling 
the credits for a single question, thereby doubling the effect 
ОЁ measurement errors in the question. The availability of 
Several different questions on each concept makes it possible 
to include more than one for any that are considered more 
important, These considerations apply whether items are 

eing constructed for a single examination or for a central file. 

The check list, Types of Questions, їп Table 2 is not in- 
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TABLE 3 
Criteria for Constructing Multiple-Choice Items! 


I. General Validity 


1. 


2. 


П. Item Content 
1 


IIL Item Structure 


Can we readily predict that those candidates who know the mm 
would, on the average, be better qualified for the purpose at hand thà 
those who do not? 

Is the item thought-provoking, rather than calling merel 
information? 


y for scraps of 


uestion—not 


| Е ЕГЕР 
Is the content of the item important enough to justify qd know the 


so specialized that only a few highly selected experts coul 
answer? lated to 
Does the subject matter of the item appear to be reasonably rela 
some type of activity appropriately covered by the test? йб 
Are the subject matter and the phrasing of the item suc ОЗ р 
tional antagonism will be aroused on the part of the public and pwi merit 
date? How would the item look in a newspaper unfriendly to t r merit 
system? (Although this criterion is primarily formulated bino nee! 
system examiners, it has application in ae testing; there 1 

to antagonize students unduly in testing them. ; hav- 
Is the nitent of the item ЕЕЕ Which can be learned without 

ing actually been on the job itself? tative be 
Does the subject matter mirror a consensus of current author! z 

liefs and opinions, rather than the opinion of one individual? than one 
Could general terms be used to make the item usable in moa tiom h 
testing situation. If the principle is applicable only in one 519 

this fact been flagged by specific reference in the item? relation" 
Does the item call for a knowledge of concepts, reasons; ns former 5 
ships, rather than for mere factual information, whenever t? 
appropriate? ualification’ 
Could any part of the item, such as modifying phrases, Т оп ü 
etc, be omitted without significantly influencing the C5 

responses? 


A. Premise П сані, 
| : ta ig 
1. Is a definite task clearly and unambiguously set, 50 а бу ае 


dates work on essentially the same problem? Is it SU тот. 
that an informed candidate could E. the correct pne choices 
premise if it were written as a completion item W! por 
given? А an Шол 

2. Is the idea stated clearly and directly, with the алас prepositi? 
tant part of the statement—not buried at the enc 0 om 
in a parenthetical “which clause”? Р stitute 2 e 

3. Does the premise, combined with each choice, comatically? te 2 
plete unit of thought, both ideologically and gramm nstit" 


: t у > r, CONS some" 
4. Does the premise, combined with the right ans require ? 
definite and true concept? Is the premise phrase й tive 
thing more (the correct choice) for completeness‘ 
5. If the question is stated negatively, can 1t b 
q g y he candidate bee? 


1 These criteria, stated in terms of merit system examining, 


terms? If not, has the attention of t d оё 

toward the negative phrasing of the question! than the B f 
6. Is the premise inen | to ask for the best rather а test © 

answer whenever possible? А mes an 
7. Is the premise stated so complexly that the item Вее er Mw 

whatever is taken to understand a complex pret m гч. 

test of knowledge or reasoning? If so, is this W d app 

be mace 
can 


cable to other testing situations by appropriate changes in wording- 


——— — - ~~ 0 NN 


| 
| 
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TABLE 3 
Criteria for Constructing Multiple-Choice Items 


8. Does the premise avoid all unnecessary content which might give 
away the answers to other items? 

9. If there are other answers conceivably as good as the intended 
answer, is the premise limited by the phrase, “of the following”? 


B. Correct Answer 
l. Is there one and only one correct answer to the problem as set by 
the premise? If the intended answer is merely the best of those 
given, is this indicated clearly in the premise? 
Do competent authorities agree on the correct answer? 
Could the candidate distinguish the correct answer from the incor- 
rect answers without having read the premise? If so, the item is 
in need of revision. 
4. Does selecting the correct choice require ed 
standing of the concept rather than mere recall or recognition? 
5. Is there no possibility that a candidate may select the correct re- 
sponse simply because it is the only one containing the same words 
* or phrases as the premise or because of other external characteris- 


tics? 


a 
SS 


a real or reasoned under- 


С. Distracters 

l. Are the distracters such that a person likely to be inferior on the 

job will think they are correct? Is each distracter so plausible that 
someone not knowing the correct answer will choose it? | 

2. If there are popular misconceptions in the field, are the distracters 


designed to attract candidates holding those misconceptions? Can 
distracters be used to make 


familiar or stereotyped phrasing in the ters Е 
them sound plausible? Сап words or phases similar to those in the 
premise be deliberately planted in the distracters to give them added 
plausibility? Can this increased plausibility be achieved without 
distracting too many of the qualified candidates as well? Are 
specific determiners, such as “always” or “never, 
\ 3. Are are distracters, as well as the intended answer, related to the 
Premise, both ideologically and grammatically? : 
4. Are there no distracters so nearly correct that well qualified persons 
are likely to accept them and be able to defend their answers? 
5. Can the question be answered by someone who knows, not that the 
intended answer is correct, but that the, incorrect , choices are 
Obviously wrong? If so, does the item remain at the difficulty level 


originally intended? i i 
6. Do all the choices constitute possible answers to a single direct 
question implied in the premise? ; 
implied in the р h and complexity as the 


7. Are the distracters of about the same lengt 

Correct answer? If not, are there at least two 

the intended answer? "- 
о pronouns refer clearly to one and only one antecedent! А 

Are the choices parallel in grammatical form and in meaning? 

Are there repetitions in the choices which could be avoided by 


Putting the repeated thought into the premise? 


which are parallel with 


S 


te 
| leer to be all-inclusive nor is it intended to prescribe the 
Rat af to be used in asking any particular kind of question. 
check + It 1S'a guide to assist in formulating questions and a 
i © help insure that all of the appropriate kinds of ques- 


tons win be asked, 


CN 
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Once the concept to be tested and the kind of questions to 


be asked have been determined, there remains the task of 
framing objective questions which will accomplish the intended 
purpose. 

The task can be stated simply: To phrase a question in such 
terms that (a) all prospective examinees understand the task 
set; (b) those who have the requisite degree of knowledge W! 
give the intended answer; and (c) all who do not will give 
another answer. Fashioning such an item is more difficult than 
formulating the problem. The criteria in Table 3 are set UP 
as aids in constructing such items. They are by no meani 
original; they have been assembled from various sources ^ 
are summarized here for convenience. The criteria in the 
table were formulated for use in merit system examining; 626 
one, however, with slight changes in wording or in emphasis 
is applicable in other examining settings. These criteria sho" ч 
be applied to each item written; the item should not be con 
sidered as complete until the writer is satisfied that it meei 
each criterion. 

After the concept has been identified, the question € 
in a premise which is clear and the intended choice written 
the next task is that of writing distracters. The ideal 
tracters are those wrong answers actually given by 
sentatives of the group for which the item is inten ed 
some other group), in response to the question as 
pletion item. Since item writers do not normally hav 
access to such answers, it is up to them to project inen 
into the situation and say, “If I were one of the group 
this test is to be given and were asked this question as? 
pletion item and didn't know the answer, what response 
I give?” On the success with which the item writer C? 
project himself into the situation of such candidates 22 
dict their responses rests the value of the item from at 
standpoint. 

In presenting a completed item the item wri 
saying: “This item is to be included in a test 
group of examinees, some qualified and some ПО 
with respect to a particular objective. If we СОЧ ced, 
separate the qualified individuals from the unquali 


ter is, ке 
ive 
and & пай 
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examine the responses of each group, we would find the keyed 
answer given by a relatively high proportion of the qualified 
group and by a low proportion of the unqualified. Each of the 
four distracters, on the other hand, should be given by a high 
Proportion of the unqualified group and by a low proportion 
of the qualified.” If, after finishing an item, one cannot in all 
honesty make this prediction, the item should be worked on 
further. If this prediction can be made, it still needs empirical 
verification by observing the actual responses of the two 
8roups, “qualified” and “unqualified,” as defined by an accept- 
able criterion, 


INFLUENCE OF LEVEL OF MOTIVATION ON THE 
VALIDITY OF INTELLIGENCE TESTS 


L. D. HARTSON 
Oberlin College 


A TOTAL of 579 students entered Oberlin College during the 
Period 1941-43 for whom 107 were available from tests taken 
Previous to entering college. These data have been added to 
those for the group of 734 entering during the years 1934-40, 
Concerning whom а report has previously been made (2), 
making a total of 1,313 students. These data have been 
studied for the purpose of checking the earlier results, and 
€xamining the relationship between aptitude and scholastic 
achievement at different levels of motivation. 


Follow-Up of Study Reported in 1941 

b From these 1,313 students a total of 1,444 records are avail- 
т le, since there are 131 cases for whom two IQ figures were 

Ported. For some purposes these might have been averaged, 
ui as that would have prevented their use for comparing the 
м idity of the different varieties of test, each IQ has been 

Teated as though it represented a different student. The same 
Procedure being followed with those who became seniors, the 
records for 522 students yielded 577 IQ's. The two populations 
employed in the study are, therefore, 1,444 and 577. The 
rengs used in the computation of high-school scholarship rep- 
: Sent not the actual grades but «credit points" obtained by 
"b ru for equating different grading systems: College schol- 

ship is handled in centile scores, that for seniors representing 

"cumulative average for seven semesters. — | 
the fo relations were computed (1) between IQ's obtained [p 
ў Wing group tests: Otis, Terman, Henmon-Nelson, Na- 


“onal, uhl individuall adminis- 
mann-Anderson, from the nervi y 
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tered Stanford-Binet, and from the Ohio State University 
Psychological Examination, as one variable, and high-school 
scholarship, as one criterion; (2) between all of these tests and 
first-semester college scholarship, as a second criterion; (3) 
between the IQ’s on each of these test groups and the OSU Test 
scores. A comparison was also made (4) of the validities of 
the OSU Test scores and those obtained with each of the other 
tests, with scholarship used as the criterion at both the sec- 


ondary school and college levels. 


The Prediction of High-School and College 
Scholarship 


The results obtained with 1,444 students in general confirm 
those reported for the smaller group, as may be seen by com- 
Paring Tables 1 and 2 with the corresponding tables in the 1941 
Study - The group tested with the H enmon-Nelson Examina- 
tion again shows a higher validity than do the other tests 
yielding IQ's (Table 1), even though this group yields a lower 
Validity coefficient than do the students tested with the 
Kuhlmann-Anderson and with the Stanford-Binet, when their 
intelligence is measured by the OSU Examination (Table 2). 


TABLE 2 


Correlations between Various Tests and Scholarship 


= ———M— 
Scholarship OSU Test 


Test group N High- st-sem. Mean c 
school college 


enmon-Nelson 
ational ,, 


This fact offers some suggestion of super jority for the H a 
elson Test, although this presumption 15 not supported by 


Suffc; np : 
ficient numbers to make it statistically substantial. 


The Mean IQ of the Oberlin Group 
Increase in the size of the population studied from 835 to 
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1,444 represents a change of one point in the mean IQ, or an 
increase from 121 to 122. The range is 90 to 169, 99 per cent 
exceeding the general mean of 100. Sigma is 10.7. Comparison 
with the Terman-Merrill sigma, 16.3. (3, p. 37), shows that 
variability among the Oberlin student-body is much more re- 
stricted than it is in the general population. The average IQ 
for the group who became seniors is the same as for the entire 
group of freshmen. 


Mean College Scholarship of Students at Different 
IQ Levels 


As the number of students who by 1943 had reached 
senior standing was more than twice the number for whom 
data were available three years earlier, 577 as compared to 
253, a report is again made of first- and seven-semester scholar- 


ship of students at different IQ levels (Table 3). The table 


TABLE 3 
Freshman and Senior Scholarship of Students at Different 10 Levels 


Ist-semester college grades 7-semester college grades 
e 
IQ N % Mean Range Possible Actual % Mean Rang 
166-170 3 02 977 97-99 3 2 667 960 93-9 
161-165 1 01 480 48 0 
156-160 3 02 673 32-92 0 87 
151-155 8 06 854 45-98 2 1 500 870 $697 
146-150 21 15 704 13-98 14 8 571 700 29709 
141-145 36 25 (658 4-99 16 10 625 691 lr$ 
136-140 85 59 653 4-98 54 4 778 675 o 
131-135 17 81 598 1-99 64 49 766 50 2 1299 
126-130 210 145 556 1-9 126 80 635 328 599 
121-125 290 201 535 1-99 189 13 651 52 5 19 
116-120 294 204 443 1-99 190 118 601 42 1 2-95 
111-115 191 13.3 389 1-97 129 78 605 39 9 2-69 
106-110 123 84 336 1-97 84 44 524 32 4 2-96 
101-105 46 32 372 1-98 29 l4 4483 44 0 26-36 
96-100 9 06 280 3-42 9 5 556 3l 3 645 
91- 95 6 04 265 7-69 6 3 50.0 18 
- 90 1 б1 270 27 0 FN 

Total 1444 100.0 915 577 631 

the 

; level and 

also shows the range of scholarship at each + estat 


proportion of those in college long enough to reach senio he 
at each level. (Scholarship is indicated by centiles.) 
table shows that: 


>, 
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а; 
Ре эл the general tendency is for those of higher ТО 
азы ы etter scholastic records, the range of scholastic 
а is remarkably similar at the different levels. 
Shih ам "m between the lowest and the highest decile 
with an ТО a an IQ of 101 to that of 141. One student 
Ther th. io. 05 attained a 96 centile standing as a senior. 
is набе элү күзө: an inaccurate index of her ability 
the OSU ng the fact that she made a 71 centile score on 
ally hed oe with IQ’s above 150, 12 made exception- 
©, ; . 
Lass germ time had elapsed for 15 students whose IQ's are 
В but to become seniors. Eight of these have obtained the 
ойр уел in only four cases was this achieved in the normal 
Ts period. 
Sie: a of selective persistence is evidenced by 
Of the 4 on upper with the lower half of the distribution. 
Яв cre с h IQ’s above 120, 315, or 67.3 per cent, persisted, 
Point, Tha with 262, or 58.6 per cent, of the 447 below that 
Standard ratio of the difference in these proportions to the 
error of the difference is 2.74. 


Relationship between Aptitude and Achievement 

at Different Levels of Motivation 
hat one of the basic 
lastic aptitude test 
degree of moti- 
After comparing 


rt, Fei Tsao (1) 


I H 
ner о of common observation t 
Scores and : oe correlations between scho 
Vation mal gn olarship is no higher is that the 
Severa] form: € from student to student 
ав pro ulas for computing an index of effo 
posed the one which follows: 
E 
in which «Е» e i І Predicted E 4 n 
redicted E» 18 an individual's scholastic grade score, and 
is the value of the scholastic score which might 


Predi З 2 

the а from his aptitude test score when one employs 

equals cd regression equation. If a student's grade average 
is predicted average, the value of the FQ is 100, this 


inde 
X re ü 
Presenting normal effort; 


e 


if his achievement is higher 
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than his predicted achievement, the value of the FQ is more 
than 100; and vice versa. This formula was used in computing 
Effort Quotients for the 1,444 population, using the 10 and 
OSU Test scores as independent and comparable bases of cal- 
culation, along with the high-school scholarship, for the purpose 
of predicting first-semester college scholarship, and for the 577 
who had become seniors, calculating the FQ from the same two 
intelligence tests, but now employing first-semester scholarship 
for predicting scholarship for seven semesters. ' 


Correlations between Aptitude Test Scores and First- 
Semester Scholarship at Different FQ Levels 


Table 4 tabulates for each of the FQ levels the correlation 


TABLE 4 


] ter 
Correlations at Different FQ Levels between Aptitude Test Scores and First-Semesté 
Scholarship, when FQ is Computed from High-School Scholarship and 
IQ, (2) OSU Test Scores. 


А il 
35i r Scholarship centile т Scholarship cent 
evels using — using 
105 N Mean OSU N Mean 
3 
130 & above 264 125 61.56 338 101 5148 
120-129 438 198 64.39 610 155 64.67 
110-119 541 252 61.39 692 304 50.34 
100-109 465 269 49.12 642 292 42.34 
90- 99 244 223 42.85 `483 199 4677 
80- 89 373 134 43.51 549 164 2922 
70- 79 279 95 28.89 469 97 37.55 
60- 69 245 75 34.87 518 51 23.05 
Below 60 14 73 18.97 385 82 50,00 
Total 358 1444 50.00 566 1444 é 
. ship: 
between the aptitude test score and first-semester scholar ari- 
together with the numbers and means, permitting 2 oe are 
ces ^, 


eon between the results when two different aptitude ind! eit 

used, the IQ and the OSU Test scores. The predictive 10 G 

this case is computed from high-school scholarship. the 
These data indicate (1) that as the correlation between 

IQ and first-semester grades is considerably lower than SU 

corresponding correlation between scholarship and the olds 

Test score (.358 as compared with .566) this therefore ider- 


2 EA. +. consi 
true at the different motivation levels. (2) There is СО 


Чоп of the F 
Q first-semester college scholars 
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Vice between the size of r at the different levels. For 
541: am "eo employing the IQ, the range is from .145 to 
Б "dim = employing the OSU Test score, the range 
sien i to .692. (3) The level where the relationship be- 
The ec I and scholarship is closest 1s 110-119. (4) 
"il т ere the students make the highest average grades 
below th каш the highest FQ, but is one or two steps 
average e highest. (5) However, the lowest grades on the 
(6) It is ts made by those with the lowest effort indices. 
ion pes so at the level of low motivation where the correla- 

een aptitude and achievement is most out of step. 


С Я 
orrelations between Aptitude Scores and Seven-Semester 
" Scholarship at Different FQ Levels 
able 5 is comparable to Table 4, except that in the calcula- 


Correlatio : TABLE 5 
militer e Different FQ Levels between Aptitude Test Scores and Seven- 
and cholarships, when FQ is Computed from Ist-Semester Scholarship 
(1) 10, (2) OSU Test Scores; with Mean Scholarship and 
Percentage of Persistence 


p — 

FQ levels "- Scholarship centile Es Scholarship centile Per centage 

170-0; l1 Im N Mean OSU N Mean persisting 
15-169" pe 76 726l 4177 3 en A 
150-149 618 79 69.93 630 ss 6865 2 
10-129 :601 & 6184 763 22 6361 7 
20-109 645 ПШ nn m ш Wu 78 
ene 05 661 64 4956 633 78 4537 6 
30- 69 244 &à 3363 512 T 3483 6 
B4 49 188 a ita am % S 59 
elow 39 005 9 ft 22 к Ub 53 
Total 1015 © qe m dis 27 
;; 5$ 6 


32 sm 500 4% 
hip, rather than 


1 
ported scholarship, is used in the prediction formula. Again 
Score Jr de 1s made of the correlations between aptitude test 
Seven se achievement (in this instance cumulative grades for 
Tun AT ped at the different FQ levels. | (1) the 7's agam 
n the Os erably higher where the computation of FQ is E 
Extends U Test than when the IQ is employed. (2) The FQ 
Over a wider range at both extremes than is the case 
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when predicting first-semester grades, reaching from below 30 
to over 170. (3) The range of 7’s at the different motivation 
levels is even greater than was found in the case of the first- 
semester grades. Where the IQ is used it extends from .005 
to .661, and where the OSU Test score is used in computing 
the FQ, the range is from .210 to .763. (4) The correlation 
between aptitude and achievement is again closest at a point 
below the highest level, although it is nearer the upper limit 
than is true in the case of freshmen. (5) Where the FQ i$ 
based upon the IQ there is a perfect positive correlation be- 
tween degree of motivation and mean cumulative scholarship; 
but where the FQ is based upon the OSU Test score the highest 
grades were made by those with motivation indices of 110-129. 
From this high point the grade curve slopes regularly in bot 
directions. (6) By the time the students with the lowest leve 
of motivation reached senior status their grade average Was & 
12 centile. The literal grade equivalent of а 12 centile average 
for seven-semester cumulative scholarship is a low С whereas 
the letter equivalent of the 73 centile is “B.” (7) Sufficient 
time had elapsed to make it possible for 915 out of the origina 
freshman population of 1,444 to have become seniors. The m 
column in Table 5 indicates the proportion at each FQ leve 
that persisted into the senior year—FQ being computed re 
OSU Test scores and first-semester scholarship. The large 
proportion of those persisting occurred at the 110-129 lereh 
where the per cent is 78. At the lowest motivation level t А 
elimination is 73 per cent, and at the highest level it is 6 ii 
cent. (The elimination represented by these figures 15 som 
what higher than normal; 38 men from the most re е 
had left college to enter the armed forces. Their Fi ie 
however, distributed over the population range in such cha? : 
fashion as not to disturb the general averages.) (8) Par Ы 
total group the correlation between FQ and scholarship nd 
seven semesters is .702, when FQ is computed from 142 


559, when FQ is based on OSU Test scores. 


Relation between FQ and Phi Beta Kappa FO 
betwee? 


cent С asses 


For the purpose of discovering the relationship 
and the highest level of scholastic attainment, Table 


6 was PI^ 
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TABLE 6 
Proportions at Decile Levels making Phi Beta Kappa when Students are Grouped 
according to 1) First-Semester Grades, 2) OSU Test Scores, 3) 10 з, and'4) 
FQ's (derived from OSU Test Scores and Ist-Semester Scholarship). 


Ist-se- 


z OSU z 
PBK PBK 10% PBK ЕО РВК 


Decile mester Test 


levels pides N % stones N * N N * N N X 


mp 39 6 10 
4 6 4 3 66 6 22 35 6 

3-9 pth е Uau E TN 

Bi. 0 61 4 7 и п 20 409 3 6 7 30 39 

н 65 1 2 7" 5 7 62 16 26 61 15 25 

"wi 7 0 0 5 2 4 S$ 3 66 4 6 

31- 50 7 б 1» 2 5 4 5 5 3 
2110 8 61 д л 2 167 
1]. 20 56 &g 116 2 3 9 
0 48 a i535 1 2 5 
1-10 ів 65 46 1 2 45 


Pared to show the proportions elected to Phi Beta Kappa at 
different motivation levels, as compared with the proportions 
from equivalent decile levels of (1) first-semester scholarship, 
(2) osu Test scores, and (3) 10. Phi Beta Kappa election 1s 
usually granted to the upper eighth of the class. 
, Table 6 shows that final scholastic honors can be best pre- 
icted from freshman grades, 69 per cent having ranked in the 
JIBhest tenth of the class as freshmen, whereas the correspond- 
ng figure for the OSU Test is 66 per cent, and for the IQ, 35 
Percent. This might have been anticipated from the fact that 
i © correlation between first-semester scholarship and cumu- 
Ative seven-semester scholarship (which includes first-semes- 
ea 8rades) is .805, whereas the validity coefficient for the 
w. Test is 559, and for the IQ it is but 322. PBK election 
ав obtained by a few students who ranked as low as the g 
м in the test distribution. In terms of FQ, ч 
7i buting the largest proportion to PBK membership 
» from which level 39 per cent were elected. This corre- 


s 
nun. approximately to the 127-140 FQ group. 


Summary " 

577; A total of 1,444 IQ scores are available for PW. 
is Ог seniors, these having been derived from the follo g 
755: Otis, Terman, Henmon-Nelson, National, Kuhlmann 
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Anderson, and Stanford-Binet. Each student was tested with 
the OSU Psychological Examination. 

2. When the correlation figures are compared with those 
obtained with the smaller population, concerning whom а 
report was made in 1941, no significant differences appear. 

3. The mean IQ of the Oberlin group is 122, the range: 90- 
169, and sigma: 10.7. The mean for those who became seniors 
was also 122. 

4. Cumulative scholarship for seven semesters ranges from 
the lowest to the highest decile at each level from an IQ of 101 
to 141, and 8 students achieved the A.B. whose IQ’s were below 
101, although in half of these cases this required more than 
eight semesters. Mum. 

5. Some selective elimination on the basis of IQ is indicate 
by the fact that 67.3 per cent of those with IQ's above 1 
became seniors, as compared with 58.6 per cent of those wit 
IQ's below 121. The comparable figures for the upper д. 
lower half of the OSU Test distribution аге 68.9 and 57.3 iie 
cent. The FQ is a still better basis for predicting persiste” i 
with 73.6 of the upper half as compared with but 52.5 per c° 
of the lower half continuing into the senior year. 

6. The correlation between aptitude test scores and f К 
man scholarship ranges from .145 to .541, when FQ is compu? 
from IQ; from .385 to .692, when FQ is computed from 
Test score. When the correlations are made with cumul 
seven-semester scholarship, the range is still greater. Test 
the IQ is used it extends from .005 to .661; where Os 
score is used it ranges from .210 to .763. | rship 

7. Although the lowest correlations and lowest scho 1 an 
correspond to the lowest FQ's, the highest correlations the 
highest achievement are found at levels somewhat below 
highest FQ’s. | 

8. It is the 127-140 FQ level which contributes the 
proportion of students elected to Phi Beta Kappa. 


resh- 


ative 


argest 


Conclusions 


E : in 
Data from this study confirm the results obtained : for 
with a smaller population to the effect that it is poss 
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a student with an IQ as low as 91-95 to obtain an A.B. in 
Oberlin College. The fact that this achievement is apt to 
require more than the usual number of semesters emphasizes 
the role of determination and directionality. Analysis of the 
relationship between motivation and achievement shows, how- 
ever, that the highest grades are not usually made by those 
with the highest effort indices, but by those within the 56-85 
centile range. Because of the method by which the FQ index 
1$ computed, the student with an IQ of 100, who makes average 
grades, obtains a higher index than the one with an IQ of 170, 
whose achievement is commensurate with his ability; but the 
former is much less apt to achieve Phi Beta Kappa distinction. 
The study also shows that when students with low effort indices 
are grouped, the aptitude test scores give little or no basis for 
Predicting their scholarship, whereas with an able group, who 
are well motivated, the correlation may be quite high. In the 
Case of the OSU Test, the difference is represented by 7’s of 
210 and 763. 
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STATISTICAL INTERPRETATION OF SYMPTOMS 
ILLUSTRATED WITH A FACTOR ANALYSIS 
OF PROBLEM CHECK LIST ITEMS 


STANLEY S. MARZOLF anp ARTHUR HOFF LARSEN 
Illinois State Normal University 


‚ Benavior symptoms are often found to occur in conjunc- 
tion more frequently than can be accounted for by chance. 
When the behavior symptoms are of the maladjustive sort it 
15 customary to refer to their conjunction as a syndrome. 

ough belief in such syndromes is supported by clinical ob- 
Servation, this same sort of observation yields the conclusion 
that many cases of maladjustment cannot be considered typical 
ч any recognized syndrome. Furthermore, observation reveals 
that it is difficult to specify an invariant conjunction of symp- 
toms by which a syndrome may be defined. Elsewhere (3) it 
Sa peoa suggested that these situations and related problems 

est be clarified by a statistical interpretation of symptoms. 
пе statistical conception which is useful is factor analysis. 

It is the purpose of this article to illustrate how syndromes 
шау be conceived of in terms of factor analysis. Viewed in this 
е | It will be seen that (1) atypical cases are not inconsistent 
i the recognition of syndromes, and (2) a syndrome isa 
HI tendency" concept which cannot be, and is not neces- 

Чу defined as, an invariant conjunction of symptoms. 
woe Mooney Problem Check List (4) was given ur 
ho T classmen in Illinois State Normal University. This 

trument consists of 330 items about which students might 
ling 2 In using the instrument the student is asked to under- 
чей 7: items about which he worries. When this has been 

é is os further instructed to circle those items about which 
йе Seriously concerned. The ten most frequently underlined 

ms were intercorrelated. This was done with the aid of 
285 
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computing diagrams (1) which provide an estimate of tetra- 
choric т. To have included more items would have involved 
cell entries so small as to make estimates grossly inaccurate. 
A direct centroid solution was made and this was transformed 
to orthogonal simple structure and to an oblique solution. 

The extent to which the ten items were found to intercor- 
relate is shown in Table 1. Since the reliability of the check 


TABLE 1 


Intercorrelations among the 10 Most Frequently Underlined Items of the 
Mooney Problem Check List 


1 ^ 2. 3$ *$ к 6 7 & nu 
I* 2 
5 d x 
$5 33 м8 o. 
£ 37 ds 4) 
5 28 08 40 5 
6 16 43 24 05 18 
727 01 ‘2 w 15 48 2  . 
& 18 05 03 22 Б в 2 «. 
9 0 D 08 0 0D 36 2 ж ga 
00 25 хо и о 07 12. 


* The items are as follows: 

Wondering if I'll be successful in life. 
anting a more pleasing personality. 

. Lack of self-confidence. 

. Afraid to speak up in class discussion. 

. Disliking financial dependence on family. 

. Taking things too seriously. 

. Not enough sleep. 

. Too little chance to read what I like. 

. Not enough time for recreation. 

10. Restless at delay in starting life work. 


MONARO 


А е- 
all the intercor" 


list items is not known, but is probably low, or of an f 


lations are very likely attenuated. The standard err 
when the true correlation is zero, for an N of 20 
Though the data are admittedly fallible, we do not 
that they are so much so as to prevent their use for t 
trative purposes we intend. + cannot 
Though no negative correlations were obtained, 1t РУ to 
be assumed that the true relationships are positive, б 
the unreliability of the obtained 7’s. 
The second factor residuals are shown in Table 2. er and 
The B-group procedure, recommended by Holziné ut Ë 
Harman (2, p. 23), did not produce clear-cut grouP® 


he illu 
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TABLE 2 
Second Factor Residuals 

P AME 5 4 5. 6. 7. 8. 9; 10: 
1° 096 -218  .015 -007 -112 -056 038 -.038 170 
2 096 -034 -105 -133 212 -071 -092 .023 -071 
3. ~218 034 077 2149 022 -003 -044  .063 -.035 
Н :015 105 .077 —.118 -123 .074 104 -.040 -017 
& 007 -133 49 -118 003 055 -004 -.006 -046 
-112 212 022 -123 .003 =027 -196  .110 -.037 
7. 056 -071 -003 074 055 —027 134 -130 -.085 
8. 038 —092 046 104 -004 —196  .134 —089 -.017 
ia -038  .023 .063  —040 -.006 .110 -130 -.089 060 

0. 170 —071 ~035 -017 -.046 -.037 -.085 -017  .060 


. = 3 
Numbers refer to items in footnote to Table 1. 


Was possible to arrange the items in a rough approximation of 
two groups. In Table 1 the first five items constitute one 
group and the last five a second. Since a group is defined as a 
Set of items within a correlation matrix which correlate more 
With each other than with other items in the matrix, a group 
тау be said to constitute a syndrome. How definitely the 
Items of these groups should be thought of as a syndrome will 
be developed later after the results of the analysis are given. 
he fact that the items within a group show correlations with 
Some items not in the group illustrates one reason why syn- 
dromes are not always clearly defined. If a greater number of 
items had been included in the matrix, the items which do not 
fit well in either of these two groups would probably be found 
to associate themselves with some of the added ones to consti- 
tute a third group or syndrome. 
In order to illustrate the app 
the interpretation of symptom intercorr с Å 
to these data. A direct centroid factor solution was obtained. 
nly two factors seemed to be of sufficient reliability, as judged 
Оп the basis of the size of the residuals, to merit consideration. 
© centroid factors were transformed to orthogonal simple 
Structure, and the result is shown in Table 3. 
... By comparing the loadings of the items on the two factors, 
» Will be seen that the factor axes pass through or мегу close 
5 the points representing items three and seven (see Figure 1). 
n fact, the rotation from the centroid axes to orthogonal 


licability of factor analysis to 
elations, we turn now 
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TABLE 3 
Orthogonal Simple Structure 


Variable* Mı М. Communality 
1. 797 101 645 
2. 552 142 325 
3. .750 1001 563 
4. 431 118 200 
5. 335 197 1151 
6. 290 405 248 
1. 002 633 401 
8. 101 610 382 
9, .022 .601 .361 
10. 073 213 050 
3.326 
Per cent of 


total variance 
19.0 14.3 33.3 


* Numbers refer to items in footnote to Table T. 


simple structure was determined by these two items. Tine 
procedure follows Reyburn and Taylor (5), who criticize 
Thurstone’s criteria for obtaining simple structure. It is Ше 
contention that hypotheses concerning the probable nature © 
the factors should be used in determining the rotation, for t9 
do so is just as defensible as to use hypotheses concerning the 
nature of factors as a basis for selecting items for intercorrela- 
tion. In the present instance, our choice was not one of com- 
plete freedom, for a meaningful analysis could not have bee? 
achieved by any arbitrarily chosen angle of rotation. 

Consideration of the factor structure as shown in Table : 
indicates that items one to four are most closely related to the 
first factor and that items seven, eight, and nine are most 
closely related to the second factor. Thus one syndrome eon" 
sists chiefly of confusion or worry ab 
a more pleasing personality, lack of 
of speaking in class discussions. 
worry about lack of sleep, 
reading, and lack of opp 
These two syndromes will b 
students as representing 
come to their attention. 
underlined items contain 
with expectation. 


out future success in же 
self-confidence, and we 
The second syndrome involve 
lack of opportunity to do preferré 
ortunity to engage in recreation. 
€ recognized by counselors of golep” 
a sizeable proportion of those W 
The fact that the ten most frequen 
these two syndromes is thus in acc? 


а 2: dip» 
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The three items, five, six, and ten, which are not so defi- 
nitely characteristic of the two syndromes, would probably 
align themselves with other symptoms if the correlation matrix 
included more items. This is most likely true for items five 
and ten, since the total variance attributable to these two 
factors is small. It is of course possible that with a larger 
Correlation matrix the groups here identified might not exist. 
However, all clinical observation makes this possibility seem 
unlikely, The magnitude of the loadings would doubtless 
change, but it seems likely that two such clusters would be 
found and that it would be possible to pass axes through them 
Without doing violence to the data. 

The distribution of the loadings on these two factors sug- 
&ests the possibility that a bi-factor solution would be the best 

€scription of the data. However, since there were but two 

Broups, Such a solution was not made. It is doubtful if the 
Solution, had it been made, would have added much to the 
interpretation, and at any rate the illustrative purpose of this 
“iscussion сап be achieved without it. 

We shall now turn to the question of causal factors. The 
ынты contains one item, namely, worry about lacking 
ums т ence, which is apparently more basic than the others 
ааз M. E identify the є ne : the root 8 the 
Ne. i he iiem to us er this as the causal actor 

Bose s the first syndrome comes from psychological 
m other than that presented in these data. It is an 
ased largely on case studies. 
ifa на а from clinical observation is even more necessary 
isto be " e causal factor responsible for the second syndrome 
^ sels entified. Such observation suggests that the second 
reae ie result from a lack of integration. Students 
hierarchy pe annis to run in all directions at once. A 
that ee erences is all too often absent with the result 
engage in E 5 complain of not being able to study, read, or 
creation to the extent that they would like. 
Чоп oF aoe correlated factors leads to a considera- 
located =i que so ution. The correlated factor axes were 
S to pass near item one and through the cluster 
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formed by items seven, eight, and nine. The angle dein 
tion between these correlated reference vectors n. oe 
correlation of .259. The location of the axes and t eir eT 
to the orthogonal reference vectors is shown in Figure 1. 


TABLE 4 
Correlated Factor Solution 
Structure Pattern 
Variable* Gi 
губ; ея ея 
1 803 198 .806 d 
2 .566 208 .549 “107 
3 744 092 -772 70 58 
4 443 169 428 13 
5 1360 236 320 TA 
6. 344 438 248 `650 
7 ‘090 629 —078 Pi 
8. 185 618 027 14 
9. 106 ‘600 —053 508 
10. -102 .220 .048 3 


* Numbers refer to items in footnote to Table 1. 
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The interpretation of the syndromes based upon the oblique 
solution does not differ greatly from that already discussed in 
relation to the orthogonal solution. There is, however, the 
Problem of accounting for the correlation between these two 
correlated factors. 

Several plausible bases for such correlation may be sug- 
gested. Students doubtless differ in their likelihood of worry- 
ing about anything. They may also differ in their willingness 
to admit worry. Ability to identify the source of their dis- 
content may be another basis for differentiating students. 
Any one of these factors might be responsible for the correla- 
tion between the two vectors. Furthermore, lack of integra- 
tion may be a factor in producing feelings of inadequacy, and 
thus the factors would be correlated. While the basis for 
correlation cannot be identified, there are a number of reasons 
why it might be expected to exist. In fact, a general factor 
of adjustment-nonadjustment, and hence correlated factors, are 
More to be expected than are uncorrelated factors. 


TABLE 5 
Total Contribution of Correlated Factors 


С, G: 
С. 1.906 ser 
Ga ‚027 1.397 


Grand Total 3.33 


Now let us suppose that all possible symptoms of malad- 
Justment were intercorrelated and that all unreliability factors 
Were absent. In this hypothetical situation, what might we 
expect factor analysis to reveal? It seems possible that a rela- 
tively small number of factors might be discovered, each rep- 
resenting an axis passing through a cluster of symptoms. 
ae clusters would represent maladjustment syndromes. If 
um CM of adjustment were also included in this hypothet- 
DA rrelation matrix, we may conceive of bi-polar factors as 

enting the most adequate reference vectors. 
OW one of the principal values to be attained by a con- 


s ; : ns 
deration of Symptoms in the statistical framework of factor 


292 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


analysis is the manner in which it enables us to deal with the 
typical and atypical cases. 

Consider first the orthogonal solution. Suppose that we 
have an individual with very marked feelings of inferiority 
or inadequacy, in other words one who “has” a large amount 
of the first factor. Since the second factor is uncorrelated 
with it, this individual may have much, none, or very little of 
the second factor. Assuming the factor loadings shown 1m 
Table 2 to be reliable, we would expect such a person to com- 
plain of worry about his future and the inadequacy of his per- 
sonality, to be lacking in self-confidence, and to fear to speak 
up in clas. He would be typical of the first or inferiority 
syndrome. Another individual might have no inferiority feel- 
ings but be greatly lacking in integration. This person woul! 
be typical of the second or unorganized syndrome. What 1$ 
most likely, however, is that relatively more individuals would 
have moderate amounts of both factors, that is, would feel 
moderately inferior and be moderately lacking in integrati?" 
Such persons would not be typical of either syndrome ап 
would show symptoms present in both. Thus it appears that 


B » 
we might expect that most people would not be "typ!c4 
cases. i 


a There remains the rare possibility that a given igdi 
vidual, on the basis of probability, would have large amounts 
of both factors. This person would not be typical either. 

In the case of correlated factors the chief difference 1$ Hum 
having a large amount of one factor would enhance the proba 
bility of having a greater-than-average amount of the ipod 
à circumstance which would make a clear-cut distinctio" 
between syndromes even less 
uncorrelated factors. 

Factor analyses of 


likely than in the case 


intercorrelations between most vi 
quently underlined items of the Mooney Problem Check bist 
Provide data for illustrative purposes, With these dat@ the 
feasibility and desirability of factor analysis as a framewor 
for clarifying clinical findings сап be shown. It is demo?" 
strated that though syndromes may be said to exist, it is no 
surprising that typical cases representing these syndromes 2 
not more frequently encountered. Though these data 


py 


yo 
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with the milder symptoms of maladjustment, the same prin- 
ciples may apply to the interpretation of psychoneuroses and 
psychoses. 

In addition to making the existence of the atypical case 
consistent with the concept of syndrome, it is also possible to 
understand why syndromes can scarcely be defined as an in- 
variant conjunction of symptoms. The magnitude of the cor- 
relations of the items of either of the syndromes with the factor 
assumed to be responsible for it is considerably less than per- 
fect. Therefore, even though the basic factor is present we 
cannot expect that all of the symptoms will be. Various con- 
tingencies of the environment may exist without any relation 
to the existence of the causal factor and may thus alter its 
manifestation. 

Furthermore, it is not inconsistent with the concept of 
syndrome for one symptom to be present in more than one 
syndrome; that is, to be moderately correlated with more than 
one causal factor. Item six, “taking things too seriously,” may 

€ an example of such a possibility. 

A syndrome must be considered as a central tendency con- 
cept rather than an invariant conjunction of symptoms. Syn- 
dromes are recognized clinically because individuals with great 
amounts of the various causal factors do exist. These more 

ramatic instances are easily recognized. Itis not inconsistent 
with their identification for us to find many other cases which 
are not typical. 
. The data also lend support to clinical observations regard- 
ing the frequency of certain behavior patterns in students. 
he evidence is most certainly not conclusive. The statistician 
Will be inclined to doubt its validity and the clinician may feel 
ane evidence is unnecessary. On this matter we can only 
rse Wolfle’s (6, p. 40) statement that closer cooperation 
tween clinician and factor analyst is desirable. 
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NEW TESTS OF 1944-1945 


California Test of Personality: Primary and Adult Series, by Louis P. Thorpe, Willis 
W. Clark and Ernest W. Tiegs, 1944. Scores are obtained for two main fields, 
self-adjustment and social adjustment. Each of these fields is further divided 
Into six areas for which scores are obtained. Tests are in packages of 25, with a 
manual of directions. Primary series, Form A, per 25 tests, $1.00; Primary 
Series, Form B, per 25 tests, $1.00; 5c per copy for tests when bought in smaller 
quantities; Primary specimen set, 25c. Adult series, Forms A and B, same prices 
as for Primary. Published by the California Test Bureau, 5916 Hollywood 
Blvd., Los Angeles 28, California. 


ERC Stenographic Aptitude Test, by Walter L. Deemer, Jr., 1944. Time, 50 min- 
utes. The purpose of this test is to estimate the probable performance ot pupils 
studying shorthand. There are 5 subjects: speed of writing, word discrimination, 
Phonetic spelling, vocabulary, sentence dictation. Package of 25, $3.50; specimen 
Set, 35c, Published by Science Research Associates, 228 S. Wabash Ave., Chi- 
Саро 4, Illinois. 


Poust-Schorling Test of Functional Thinking in Mathematics, by Judson W. Foust 
and Ralph Schorling, 1945. Time, 45 minutes. This test is intended to measure 
the power to deal with mathematical relationships. It is usable in the 9th grade 
and above. Sold only in packages of 25 copies with manual and key at $1.35 
Per package; specimen set, 15с. Published by World Book Company, Yonkers- 
on-the-Hudson, N. Y. 


General Clerical Test, 1944. Time, 50 minutes. This is a test which has been de- 
Signed as a general and differential test for use in selecting persons for all types 
of clerical worl. It is for high school and above. Sold in packages of 25 with 
manual and key, 1-3 packages, $3.25 each; 4—39 packages, $2.85 each; 40 or 
more packages, $2.60 each. Рег copy when package is broken, 15c. Specimen 


Set, 30c. Published by the New York Psychological Corporation, 522 Fifth Ave., 
ew York 18, N. Y. 


,N. 

Goldstein-Scheerer Test of Abstract and Concrete Thinking, by Kurt Goldstein and 

artin Scheerer, 1945. These tests are for the use of psychologists, psychiatrists, 

and neurologists working with patients who have brain injuries. ‘They are de- 

Signed to measure both quantitatively and qualitatively the impairment of the 

unction of the brain with reference to abstract and concrete reasoning. The 

monograph contains full instructions for the administration and interpretation 

of the tests, Three of the five tests are available: the Goldstein-Scheerer Cube 

est, the Weigl-Goldstein-Scheerer Color-Form Sorting Test, and the Goldstein- 

Scheerer Stick Test. Cube Test: 2 design books, $4.75 per set; Kohs blocks, 

-00 Per set of 12; record forms, sold only in packages of 50, $2.75. Stick Test: 

Drastic sticks of 4 lengths, $2.00 per set; record forms, including mimeo- 

graphed supplement to the monograph, sold only in packages of 50, $2.75. 

Omplete materials, all three tests, including a copy of the monograph, “Ab- 

Stract and Concrete Behaviour,” $21.50. Published by the Psychological Cor- 
Poration, 522 Fifth Avenue, New York 18, N. Y. 


How ; 3 a eer C 
Supervise? by Quentin W. File. No time limit. Norms are given in percentile 
z E and Standardized scores. This test is considered as a measuring instrument 
| © supervisors’ knowledge and insight concerning human relations in in- 
295 
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ni i ts 
dustry. It is designed to aid management in obtaining a clearer peers 
supervisors’ understanding of the more important фый aspects Журе 
There аге 3 sections: Supervisory Practices, Company ae pu key. l9 
Opinions. Forms A and B. Sold in packages of 25 with manua d Memes 
3 packages, $2.00 each; 4 to 39, $1.65 each; 40 or more, $1.50 each; si а 02 КИН 
10c. Specimen set, 30c. Published by the Psychological Corporation, 522 
Avenue, New York 18, N. Y. 


: ; G 
Inventory of Factors GAMIN (Abridged Edition), by J. P. Guilford and H. Z 
Martin. This test is designed to measure the following po uations 
istics: G, general pressure for overt activity; A, ascendency in socia ie as Op" 
as opposed to submissiveness; M, masculinity of attitudes and Did Santee 
posed to femininity; I, lack of inferiority feelings; and N, lack of дегу СЕ itor He 
ness and irritability. Supplement to the manual and scoring keys En scoring 
printed shortly, 100 copies, $10.00; 50 copies, $6.00; specimen set wi ume Sup- 
keys, 25c; specimen set with scoring keys, 50c. Published by the Sherida 
ply Company, P.O. Box 837, Beverly Hills, California. 


is self- 

Johnson Temperament Analysis, by Roswell H. Johnson, 1944. The form И юге 
administering and ordinarily requires 40 to 50 minutes. The following 

are obtained: nervous, 

sive, critical, and self- 


i Per 
f the specimen set is 25c postpaid. ic 


A ets, 
» $125; 10с each in smaller quantities. Response record shee 16 


each; analysis profiles, 1c each. Published by the California Test Bureau, 
Hollywood Blvd., Los Angeles 28, California. 


a eae 944. 
Mellenbruch. Aptitude Test for Men and Women, by Paul L. Mellenbruch, 1 


s е 22:2 1 ^ of the 
Time, 35 minutes, This test is designed to discover the degree and litis ions. 
mechanical trainability of men and women applicants for mechanical р 


x гаро. "Icants ` schoo! 
It is also intended as a counseling aid for selecting junior and senior hene ude 
students who will profit from industrial arts training. Scoring key 15 1 
free with each order of 2 


re 
" 5 or more booklets. Published by Science Rese? 
Associates, 228 Wabash Ave., Chicago 4, Illinois. 
way 
Minnesota Multiphasic Personality Inventory, Group Form, by Starke R. Наа 
and J. Charnley McKinley.” Time Tequired is from 30 to 90 mmo w 
individual form, Published earlier, has been adapted to group form for Ч 
sion, 


ten- 
à it 

p ; Paranoia, masculinity, hypomania, validity» Sed. 
dency to lie and tendency to g 


d manual, $7.50, I.B.M. answer logic? 
F in small Sieg: ished by the Psycho 
Corporation, 522 Fifth Ned EHE i plahed У 


. e — Bá 6 Е Lee 
Occupational Interest Inventory—Intermediate, Form A, devised by Edwin. fol- 
and Louis P. Thorpe, 1944: or junior high-school students to adults. , aas 
owing scores are obtained: personal-social, natural, mechanical, busin Pack- 

arts, the sciences, verbal, mani 


ipulative, computational, and interest level- * "Cor, 

age of 25 tests with manual, $1.75; 8c cach d smaller quantities; spe Los 

Pe ubublished by the California Test Burgi isi Hollywood B 

Angeles 28, California. i No 
Occupational Preference Inventory, by Paul P. and Ralph T. Brainard, юр, ite 

time limit. This blank replaces the Brainard and Stewart Specific Inte are d 

ventories, which will go out of print shortly. Occupational preferences 27. 

vided into 28 Occupation 


: i 
р ч & B 
е А al sections, and the 28 sections are combin' panicab 
major occupational fields: 


T 


4 


з» = 
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professional, esthetic and scientific. Booklets сап be reused. Inventory booklets 
with manual: 1 to 9 copies, 25c each; 10 to 99, 22c each; 100 or more, 20c each: 
Manual is 25c extra when fewer than 10 are ordered; record forms are sold only 
in packages of 50. 1 package is $3.75; from 2 to 19, $3.50 each; 20 or more, 
$3.00 each; per copy, 10c each. Specimen set is 50c. Published by the Psycho- 
logical Corporation, 522 Fifth Avenue, New York 18, N. Y. 


Personal Audit (Revised), Forms LL and SS, by Clifford К. Adams, 1945. No time 
limits. Form LL consists of 9 parts, each part containing 50 items. This form 
can be used with adults having the equivalent of a grammar school education or 
with senior high school students. The first six parts of Form LL are published 
separately as Form SS. Form SS is recommended for use with junior high 
School students or in other situations where testing time must be curtailed. 
Scoring requires no keys. The fields for which scores are obtained are: serious- 
ness-impulsiveness, tranquility-irritability, frankness-evasion, stability-instability, 
tolerance-intolerance, steadiness-emotionality, persistence-fluctuation, content- 
ment-worry. Form LL: package of 25 tests with manual, $3.75; specimen set, 
35c. Form SS: package of 25 tests with manual, $2.50; specimen set, 25c. 
Published by Science Research Associates, 228 S. Wabash Ave., Chicago 4, 

inois. 


Pintner General Ability Tests: Non-Language Series, by Rudolph Pintner, 1945. 
hese group tests are designed to measure mental functions independently of 
word knowledge and facility, and may even be administered in pantomime with- 

out the use of language, using special directions which can be obtained оп re- 
Quest. The series is to consist of two batteries: the Intermediate Test for grades 

4 through 9, and the Advanced Test for grades 9 and above. The Intermediate 

est is now available. Package of 25 tests with manual, $1.80; specimen set, 

SU Published by the World Book Company, Yonkers-on-the-Hudson, New 


ork. 


P-L-S Journalism Test, by George H. Phillips, Harry Levinson and H. E. Schram- 
mel, 1944. Time, 40 minutes. This test is designed for use in high school and 
college classes which have completed a first course in journalism. | Percentile 
norms are given. Per package of 25, directions and key included, $1.00 f.o.b. 
Emporia; $1.15 postpaid. In quantities less than 25, test 5с, directions 5c, speci- 
men set, 15с. Published and distributed by the Bureau of Educational Measure- 
ments, Kansas State Teachers College, Emporia, Kansas. 


Reading Achievement Test—Intermediate Test: Forms A and В, by Donald D. Dur- 
rell and Helen Blair Sullivan, 1944. Time, 30 to 45 minutes. These forms of 
the Reading Achievement Test are designed for grades 3 to 6. Sold only in 
Packages of 25 with manual and key. Form А or B, $1.55. Specimen set, 55с. 
Published by the World Book Company, Yonkers-on-the-Hudson, New York. 


Schrammel-Otterstrom Arithmetic Test, by H. E. Schrammel and Ruth Otterstrom, 
1945, Time, 50 minutes. This test consists of two divisions: Test II for grades 
4, 5, and 6; and Test II, for grades 7 and 8. Percentile norms are provided for 
interpreting the scores for each part and for the entire test, both mid-year and 
end-of-year testing. Per package of 25, directions and key included, 60c f.o.b. 
Emporia; 75c postpaid. In quantities less than 25; test 3c, key 3c, directions 5c. 
Specimen set, 20c. postpaid. Published and distributed by the Bureau of Edu- 
cational Measurements, Kansas State Teachers College, Emporia, Kansas. 


Stevens Reading Readiness Test, by Avis Coultas Stevens, 1944. These tests assist 
the teacher to group her beginning children for reading. They are not intended 
to displace intelligence or kindergarten tests. Package of 25 tests with manual 
and key, $1.80; specimen set, 20c. Published by the World Book Company, 

onkers-on-the-Hudson, New York. 


Survey of Mechanical Insight, designed by D. R. Miller, 1945, Time, 25 minutes. 
esigned to measure aptitude for solving the types of problems involved in jobs 
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; i es of ma- 
ў i various types О 
iring the operation, maintenance, Tepair, or байо Sy Pg each in smaller 
since Package of 25 tests with manual and key, $2.50; 
chinery. 


lvd., 
5 llywood B! 
quantities. Published by the California Test Bureau, 5916 Holly 

Los Angeles 28, California. 


i This sur- 
; $ Ti nutes, 
Survey of Object Visualization, by D. В. Miller, 1945. Time, = when its shape and 
xs requires the examinee to predict how an object will loo The test is practi 

Position are changed. Tentative Percentile norms are Сава Package o 7 
cally self-administering and the scoring is complete y pa $150; 7c a сору ig 
tests, including a manual of directions and scoring Bien, 2916 Holly 
smaller quantities. Published by the California Test 
Blvd., Los Angeles 28, California. 


E EC тесту 1944. 
9 m d Floyd Ruch, | 
Space Relations Ability, by Harry W. Case an 10 the emp n 
Bun a This test is designed to measure the ee d ш Ко ien ү 
applicant to perceive rapidly and accurately the UEM T 25 tests, II 
Space. Tentative percentile norms are furnished. Packag 


blished 
- t, 25с. Ри, 
l of directions and scoring key, $2.00; specimen set, eles 28; 
i the California "Test Bureau, 5916 Hollywood Blvd., Los Ang 
fornia. 


Time, 
oyee ОГ 


SL individual test 
vid Wechsler, 194 his a short indivi Sol 
2% avid Wechsler, 1945. This ion 
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Subject Tests. These tests are to determine the individual’s proficiency in spe- 
cific subjects on both high school and college levels. The tests include: 


Examination in English—High School Level; Book I: Reading пай шее 
tion of Literature and Literary Acquaintance. Percentile norms for to 
available for grades 10, 11 and 12. N 
Examination in English—High School Level; Book II: Composition. Norms are 
available. A : m 
Examination in English —College Dn Book I: Reading and Literary 
quaintance. Percentile norms for total score are aval "MR. 5 
Examination in English—College Level; Book II; Composition. Percentile 
norms are available. . E 
Examination in Business English —High School Level. 29 porre available 
Examination in Commercial Correspondence—College Level. o 
able. , б 
vailable. 
Examination in French Vocabulary—Lower Level. No norms a Р 
Examination in French Reading Comprehension—Lower Level. No norm 
available. А 
Examination in French Grammar—Lower Level. раза башт. 
Examination in French Vocabulary—U pper Leve " o noras уа d. vids 
Examination in French Reading Comprehension—Upper j 
able. S 
Examination in French Grammar—Upper Level. Ns pum unes. 
Examination in German Vocabulary—Lower Level. Ne пог! ava x onm 
Examination in German Reading Comprehension—Lower Level. 
available. r 1 
Examination in Spanish Vocabulary—Lower Level. Ne Bo eee ET. 
Examination in Spanish Reading Comprehension—Lower А 
available. р 
Examination in Spanish балш. No norms available 
Examination in Italian Vocabulary. No norms ` caves 
Examination in Italian Reading Comprehension—Lower Level. No norm 
able. d 
Examination in Italian Grammar—Lower Level. m sett Available. atte 
Examination in Business Arith metic—High School FE i No norma IS DS 
Examination in Advanced Arithmetic—High School puel: 2 S зоди RATES 
Examination in Elementary Algebra—High School Level. Per 
total score available. . Я e 
Examination in Second-year Algebra—High School Level. Percentile norms fi 
t ilable. | 
Fes ie con Geometry—High School Level. Percentile norms for total 
score available. Р 
xamination in Algebra—College Level. No norms ате 24 sce 
Examination in Plane Trigonometry—College Level. x o Sung Pay 
Examination in. Analytic Geometry—College Level. Y no: iced 
Examination in General Science—High School Level. No ea 
Examination in Biology—High School Level. No norms RERS 
Examination in Physics—High School Level. No norms r ШЕЕ UT im 
xamination in Chemistry—High School Level. Percentile 
available. " 
Examination in Meteorology—High School Level. ne anans уры. 
Examination in Senior Science—High School Level. No nori i 2 ; 
Examination in Chemistry—College Level. Percentile norms for total score 
available. 2 » J 
Chemistry Examination in Qualitative Analysis. No scaled scores provided. 
ercentile no vailable. " ? Я 
Cham stry X рыя in Quantitative Analysis. Percentile norms available. 
Examination in Organic Chemistry. Percentile norms availa le. 
Examination in Astronomy—College Level. No norms aya able. е 
Examination in Elementary Psychology—College Level. © norms available. 
Examination in American History—High School and College Levels. Percentile 
norms available. 
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Examination in World History—High School Level. No norms available. 
Examination in Civics—High School Level. No norms available. 


Examination in Problems of Democracy—High School Level. No norms avail- 
able. 


Examination in German Grammar—Lower Level. N 


o norms available. 
Examination in Business English —High School Level 


. No norms available. 


Examination in Commercial Correspondence—College Level. No norms avail- 
able. 
Examination in Business Arithmetic—High School Level. Percentile norms 
available. 
Examination in Gregg Shorthand—First Year—Secondary School. No norms 
available. i 
1 имен» in Typewriting—First Year—Secondary School. No norms avail- 
able. ч 
Тотай» in Typewriting—Second Year—Secondary School. No norms avail- 
able. 
Examination in Bookkeeping and Accounting—First Year—Secondary School. 
No norms available. | 
Examination in Bookkeeping and Accounting—Second Year—Secondary School. 
No norms available. И 
xamination in Mechanical Drawing—High School Level. No norms available- 
xamination in E T 


THE CONTRIBUTORS 


Herbert A. Toops is professor of psychology at the Ohio State 
University. He is the author of numerous articles and books in the 
elds of personnel, guidance, measurement, statistics and motivation. 
The Ohio State University Psychological Examination, developed by 
T. loops, is used in the statewide high school testing programs of 
four states. During the war Dr. Toops spent considerable time in 
ashington. For six months of 1944 he was with the Office of the 
secretary of War working on the development of a statistical report- 
ing system for civilian employees of the War Department. He also 
Spent three months in the early part of 1945 on a survey of the 
ational Roster of Scientific and Specialized Personnel. He is a 
member of the editorial board of Zducational and Psychological 
Teasurement. 


Milton M. Mandell is chief of the administrative and manage- 
ment testing sub-unit of the United States Civil Service Commission. 
‚18 previous positions have included those of personnel officer, Mate- 
rials Division, W.P.B.; assistant chief of examinations, Los Angeles 
City Civil Service Commission; and classification consultant, Con- 
necticut State Personnel Department. Mr. Mandell was co-author 
With Wallace Sayre of *Education and the Civil Service in New York 
ity,” published by the United States Office of Education. 


Major Thomas W. Harrell received his Ph.D. degree from the 
Johns Hopkins University in 1936. He joined the staff of the Univer- 
Sity of Illinois in 1936 and taught there until May, 1940, From 1938— 

he conducted test research and consulted with the Air Corps 
echnical School, U.S. Army. From May, 1940, to June, 1942, he was 
“xecutive, later Chief, Personnel Research Section, А.С.О., War De- 
Partment. Called to active duty in the Army in January, 1942, he 
Continued in the Personnel Research Section until September, first in 
15 former position and later as Air Liaison Officer from the A.G.O. 
to the Army Air Forces. In September he transferred to Headquar- 
ters of the A.A.F. where he was Classification Officer until May, 1944, 
Ince that time he has been overseas serving as Air Liaison Officer, 
q; Replacement Command, NATOUSA until February, 1945, as 
Chief, Personnel Classification Audit, AAF, MTO, July-October, 
1944, and then with Hq. 15th Air Force as Director, Manning Sec- 
tion, А-1 Division from February—July, 1945. He is now Deputy, 
Asst. Chief of Staff, Personnel, A-1, Hq. 15th Air Force. 


For five years after graduation from Stanford University in 1933, 
Tgaret S. (Mrs. Thomas W.) Harrell was research assistant for 
301 


a 


302 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


her father, Dr. E. K. Strong, Jr., in his Vocational Interest Research 
Office. From 1939-1940 she was employed in the Research and 
Analysis Unit of the San Francisco Employers Council. In Novem- 
ber, 1940, she transferred to the War Department where she was 
employed until June, 1942, first as a junior personnel technician 10 
the Personnel Research Section, A.G.O., second in the Planning Divi- 


sion, Special Service, and third as chief clerk of the Morale Research 
Division, Special Service. 


J. R. Wittenborn is assistant professor and clinical psychologist 
at Yale University. He divides his time between the department 0 
psychology, where he gives graduate instruction in statistics АП v 
clinical psychology, and the department of university health, where 4 
he is responsible for the educational occupational orientation o 
undergraduates, Since May, 1944, he has been participating 1n а 
Committee on Medical Research project for the Office of Scientific 


Research and Development. He received his Ph.D. from the Оп" 
versity of Illinois in 1942. 


an 
h the 


M. Claire Myers f Es r from 
the University of Calif rer receiving her Ph.D. in psychology +9: 


fornia, left the Insti ild Welfare J^ (б 

Сару, Hee she had been in recep dedit the Child " 
E 1 r 

Wellesley College. Years as a psychology instruct?” |, 

nical Advisory Service of the § 


ding 
one year as personnel consulta 


ocial Security Board. After spen ific 


mpensation, and has assisted i IS 
sation ed in the resea 
of the Employment Stabilizatio i i 


en R the 5 № 
dardization of Forms М and 1Ё of feted | | 


L. D. Hartson i Е f psy” 
chology at Oberlin College. Hie ures ой е department о. ре 


м 


THE CONTRIBUTORS 303 


Bureau of Tests and Measurements of the College. Dr. Hartson has 
published many articles reporting his studies on the validation of 
psychological tests and rating scales as well as the construction and 
evaluation of vocational interest questionnaires. Much of his time 
in recent years has been devoted to a study of the vocational his- 
tories of Oberlin graduates. 


. Stanley S. Marzolf is associate professor of psychology at Illi- 
nois State Normal University. Dr. Marzolf teaches graduate courses 
in the field of clinical psychology, and directs the graduate work of 
students planning to become guidance and personnel workers in high 
Schools. His several activities in the testing and guidance fields 
include chairmanship of the research committee which administers 
the university testing programs. He received his Ph.D. in psychol- 
ogy from the Ohio State University in 1937. 


Arthur Hoff Larsen is assistant dean and head of the department 

of education and psychology at Illinois State Normal University. 

€ is also director of the interviewing and testing done in the Vet- 

€rans Administration Guidance Center located on the campus. He 
received his Ph.D. from the University of Wisconsin in 1939. 


— 


MEMBERS OF THE AMERICAN COLLEGE 
PERSONNEL ASSOCIATION 


Adams, Dr. Clifford R., Marriage Counseling Service, Pennsylvania State College, 
State Coll P lvania 
dams, Lc. Was М. AGO Rémundigonisg Unit, Annex IV, Brooke General and 
Convalescent Hospital, Fort Sam Houston, Texas Р x 
iken, Prof. N. J., Director, Placement Bureau, and Associate Professor of Eco- 
nomics, State College of Washington, Pullman, Washington . 
Alexander, Miss Ruth Lenore, 8041 E. 8th Street, Bloomington, Indiana 
lbright, Miss Thelma, Dean of Students, Queens College, Charlotte, North 
C ina ‘ З " 
Allen, "Ms taal H., Head of the Division of Personnel Service, N. E. Missouri 
St College, Kirksville, Missouri | 
Anderson, MN ae Ve ‘Acting Director, studeas Counseling Bureau, 101 Eddy 
Hall iversity of Minnesota, Minneapolis, Minnesota 
Andrews’ woe. Go Director of Personnel, Florida State College for Women, 
allahassee, Florida i Wes 
Applegate, Mr. C. William, 7740 N. Ashland, Chicago 26, Illinois 
Austin, Lt, Sidney F., Quarters C Н, U. S. Naval Training Station, Newport, 
Rhode Island Я ' 
Babcock, Miss Fern, National Student Secretary, Y. УУ. С. A., 600 Lexington Avenue, 
ew York, New York : 
Bagley, Miss Leila, 1000 Morningside Drive, New York 27, New York | 
аПеу, Prof. Harold W., Director, Personnel Bureau, University of Illinois, Urbana, 
llinois i E 
Bathurst, Dr. J. E., Department of Psychology and Education, Birmingham-South- 
ern College, Birmingham, Alabama 
Belknap, Miss A, Fredericka, Director, Personnel Bureau, New Jersey College for 
omen, New B ick, New Jersey . 
Ball, Capt, Hugh. M., Classification p Counseling Section, AGO 1E936 Pentagon, 
ashington, D. C. 


Berg, Mr. Irwin A., Clinical Counselor, and Associate in Psychology, University of 


Illinois, Urbana, Illinois , 
Bergstresser, Dr. John L., Dean of Men, College of the City of New York, Convent 
; "venue and 139th Street, New York, New York | . А 
Biglow, Miss Mary D., Chairman, Advising, Stephens College, Columbia, Missouri 
illings, Mr. Earle Monroe, Personnel Director, Eastman Kodak Company, 343 
State Street, Rochester, New York AN 4 , 
Blaesser, Dean Willard W., Assistant Dean of Students, University of Chicago, Chi- 
cago, Illinois i 
Blanchard, D. (Miss) Leslie, Principal, Holmquist School, New Hope, Pennsylvania 
ishop, Mrs. Joan Fiss, Director, Placement Office, Wellesley College, Wellesley 
» Massachusetts . 
Blanding, Dean Sarah G., New York State College of Home Economics, Cornell 
niversity, Ithaca, New York 
Boyles, Lt. К. E,, 532 River Street, Hoboken, New Jersey. 
"i Dean Francis F., University of North Carolina, Chapel Hill, North 
arolina 
Bridgman, Mr. Donald S., American Telephone and Telegraph Co., 195 Broadway, 
. New York, New York А 
Bridgman, Pres. Ralph B., Hampton Institute, Hampton, Virginia 
305 


306 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


k rothy, Dean of Women, Dennison University, Granville, Ohio 
Гаа a Por Robert A., Personnel Officer, 116 College Hall, University of 
Pennsylvania, раа Fenpsylvania бы 3» Tests 
r, Mr. Paul J., niversity Avenue, Chicago 37, Illinois — ЕР 
е Мг. баң K., Director of Student Affairs and Publications Editor, 
Alabama Polytechnic Institute, Auburn, Alabama York 
Brown, Mrs. Lucile B., American Red Cross, APO 763, New York, New gee 
Brumbaugh, Dr. A. J., American Council on Education, 744 Jackson Place, Wa 
ington 6, D. C. 
Bret, Mr. Carl A., Director of Student Activities, Jefferson College, 1528 
Locust Street, St. Louis, Missouri . Ё 
Bunn, Mr, John W., Dean of Men, Stanford University, California 
Butler, Lt. John, A.S.T.P., Camp Wheeler, Georgia . Сон 
Cameron, Mrs. Marian Lippitt, University Testing Service, Syracuse Unive 
Syracuse, New York апу 9 
Camp, Miss Frances M., Director, Bureau of Appointments, State Univers 
Towa, Iowa City, Iowa ‘cont 
Cannon, Dean C. W., Consulting Psychologist, Stevenson, Jordon and Harrison 
205 W. Wacker Drive, Chicago 6, Illinois iversity of 
Carpenter, Miss M. Helen, Acting Director, Placement Bureau, Univers! 
Colorado, Boulder, Colorado 
каше Mi О. R,H 
orvallis, Oregon е it 
lark, Miss Barbara M., Assistant Director, Student Activities Bureau, University 
of Minnesota, Minneapolis, Minnesota Michigan 
Coates, Miss Dorothy R., Director, Katharine Gibbs School, 720 N. Mi 
Avenue, Chicago, Illinois Street, 
Colvin, Mr. Harold William, Y. M. C. A. National Council, 19 $. LaSalle 
Chicago, Illinois 
Compton, Dean Richard K., Hastings, Michigan hlehem, 
Соп, Dean Wray H., Dean of Undergraduates, Lehigh University, Bethle! 
а 


; f 
Congdon, Dr. Nora A., Personnel Research Officer, Colorado State College ° 
ucation, Greeley, Colorado 


Conwell, Dean H. H., Beloit College, Beloit, Wisconsin 


, kton. 
Corson, Mr. James Hunt, Dean of Men, College of the Pacific, Stockto™ 
California awr 


Crenshaw, Mrs. James L., Director, Bureau of Recommendations, Bryn i 
College, Bryn Mawr. Pennsylvania bury» 
Croft, Lt. Col. Lysle W., Наа, Int, W. D. Personnel Center, Camp Atter 
ndiana 
. te 
Crawley, Dr. S, L., Director, Department of Student Personnel, Colorado Бы 
College of Education, Greeley, Colorado xford, 
aan Richard C., Director of Student Counseling, Miami University, O. 
io i 
Cunniggim, Miss Margaret І, Dean of Women, Ripon College, Ri Wisconst 
dies „ә " , Kipon, jation 
It (ie), ion, Neo NE: Bureau of Medicine Ee Surgery Avai 
eaical Division, Nevy Dept., оготас Annex, Washington 25, D. C. 
Dalton, Glen TY. M. C. A., 600 exington Avenue, Nev York, New Grand 
avies, Mr. Roy O., Director of Personnel, Western Auto Supply Co., 2107 
Avenue, Kansas City, Missouri see 
Davis, Mr. Victor M., Director, Bureau of Appointments, University of Tenne 
Knoxville, Tenn. ў 
awson, Mr. J. D., Personnel Direct. 
erdis L., 


lege 
ead, Department of Psychology, Oregon State College, 


; ; hio . 
or, Antioch College, Yellow Springs, Ohio yje, 
Deabler, Dr. Herdi Personnel Director, North ‘Central College! Мара 
inois 


DeGolyer, Mrs. Robert S., 575 Madison Aven 
ensmore, Mr. Harvey B., Chairman of Gene 
Seattle, Washington Jose 
DeVoss, Dr. James C., Dean, Upper Division, San Jose State College, $a” 
California 


:chigan 
ue, S.E., Grand Rapids, Michige n, 


ral Studies, University of Washin 


AMERICAN COLLEGE PERSONNEL ASSOCIATION 307 


‘ 
Dillman, President Grover C., Michigan College of Mining and Technology, Hough- 
ton, Michigan А А ` 
onahue, Dr. Wilma T., Department of Psychology, University of Michigan, Ann 
Arbor, Michigan А А 
Doty, Miss Katherine S., Assistant to the Dean, Barnard College, Columbia Uni- 
versity, New York, New York А ` 
ozier, Prof. Doris, Recorder and Secretary of Placement, Mills College, Mills 
College, California . 
ressler, Miss Marguerite R., Assistant Director of Personnel, Florida State College 
for Women, Tallahassee, Florida M 3 
rought, Dr. Neal E., RCA Victor Division, Personnel Administration, Camden, 
ew Jerse ч 
Dugan, Мм Willis E., Director of Student Personnel, University High School, Uni- 
versity of Minnesota, Minneapolis, Minnesota des 
unham, Mr. Charles Vernon, Director of Student Employment, University of 
„texas, Austin 12, Texas " "€ 
Dysinger, Dr. Wendall S., Dean, MacMurray College, Jacksonville, Illinois 
dwards, Miss Marcia, Professor of Education and Assistant to Dean, College of 
Education, University of Minnesota, Minneapolis, Minnesota » 
Eldersveld, Miss Wilma, Psychological Clinic, 1027 E. Huron Street, University of 
Michigan, Ann Arbor, Michigan 4 
Elias, Miss Virginia, Director, Student Aid and Employment, Occidental College, 
1600 Campus Road, Los Angeles 14, California 
Ellingson, President Mark, Rüchcster Institute of Technology, 65 South Plymouth 
Avenue, Rochester 8, New York 4 : 
Esterly, Mrs. Virginia Judy, Assistant to the President, Scripps College, Claremont, 
alifornia К 
Evans, Mr. John J., Jr., Armstrong Cork Company, Lancaster, Pennsylvania. _ 
Evans, Miss Mary Catherine, Vocational Adviser for Women, Indiana University, 
Е loomington, Indiana А 9 m 
Wan, Mrs. Mary A. Associate Chairman, Appointments Division, College of 
, Education, Arps Hall, Ohio State University, Columbus, Ohio ; 
Failing, Miss Jean, New York State College of Home Economics, Cornell Uni- 
E versity, Ithaca, New York cd 
Falvey, Miss Frances, Adviser to Freshmen, Hollins College, Virginia 
aries, Miss Miriam, Assistant Dean of Students, College of the City of New York, 
F Convent Avenue and 139th Street, New York, New York 
arrington, Mr. E. H., Director, Placement Service, North Texas State Teachers 
F ollege, Denton, Texas - E 
"eder, Lt. Cdr. Daniel D., Bureau of Naval Personnel, Training Division, 4721 
T rlington Annex, Arlington, Virginia h È 
eldt, Miss Irene C., Director, Placement for Women, Purdue University, Lafayette, 
Felst 


ndiana 
ed, Mrs. Leona Wise, Chairman, Personnel Committee, Illinois Wesleyan Uni- 
Fig 761810, Bloomington, Illinois 4 e 
нра. Prof. Charles A., Director, Education Placement Bureau, Temple University, 
‚| "Toad and Montgomery, Philadelphia, Pennsylvania Е 
Fisk, Miss Helen G. е Director, Western Personnel Service, 30 North 
FI Raymond Avenue, Pasadena, California c 
ood, Mr. Julian J., Personnel Director, Tuskegee Institute, P. О. Вох 445, 
Uskegee, Alabama 1 
» Miss Helen E., Old Capitol, State University of Iowa, Iowa City, Iowa 
Mr. John D., Assistant to the Dean, Office of Dean of Students, University 
Gaet. Minnesota, Minneapolis 14, Minnesota A 
aetje, Miss Lucille, Director of Personnel for Women, Owens-Illinois Glass Com- 
сапра, Gas City, Indiana | 
allagher, Prof. Marie K., Chairman, Bureau of Educational Guidance, Hunter 
G College, 695 Park Avenue, New York, New York 
arrett, Mrs. Mary, Director of Admissions and Student Personnel, Bennington 
G v College, Bennington, Vermont 
ebauer, Miss Dorothy, Dean of Women, University of Texas, Austin 12, Texas 


308 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


Gerken, Mr. Clayton d’A., Department of Psychology, State University of Iowa, 
va City, 1 ssi Yol 
Gal Mis Т, Hammond, Hunter College, 695 Park Avenue, New Yor 
i k iversity of 
бйз М iin M., Clinical Counselor, Personnel Bureau, University 
inoi ‚ Urbana, Illinois 6; 
"E E. W., Director, Placement Bureau, Iowa State Teachers Colleg 
dar Falls, Iowa : : 
"M. Margaret D., Placement Director, Wilson College, Chambersburg, 
lvani MESN Florida 
Sete Gita: S., 0-582761, Box 731, 326th А.А.Е. B.U., MacDill Rad Fn 
Griffen, Miss Alice J., Director of Personnel Service, Wright Junior College, 
N. Austin Avenue, Chicago, Illinois me 
Griswold, Miss Mary Lou, Placement Secretary, New York State College of Ho 
Economics, Cornell University, Ithaca, New York inois Normal Uni- 
Gum, Miss Wanda N., Assistant Dean of Women, Southern Illinois Nor 
versity, Carbondale, Illinois А : В у York 
Gus, Mr. Charles Edward, Executive Secretary, College of Engineering, New 
niversity, University Heights, New York, New York New York 
Haggerty, Mr. William J., President, State Teachers College, New Paltz, 
ale, Dr. Lincoln B., President, Evansville College, Evansville 4, Indiana 


7 4 k 16, 
Hand, Mr. Thomas J., Consultation Service, 87 Madison Avenue, New Yor! 
New York 


Hanson, Miss Anna M. 


› Simmons College, Boston, Massachusetts College; 
Hanson, Mr. Ernest E., Dean of Men, Northern Illinois State Teachers 
De Kalb, Illinois 


Hausam, Miss Winifred M., Director 


Avenue, Pasadena, California 


" " ilton, 
Held, Lt. Cdr. Omar С., (5), USNR, Navy V-12 Unit, Colgate University, Нат 
New York 


3 " Iowa 
Helser, Dean Maurice David, Personnel Director, Iowa State College, Ames. 
Henderson, Mrs. Virginia Kinsman, P. 


venue, 
1 ersonnel Officer, 19 West Plumstead A 
Lansdowne, Pennsylvania. 


ork 
Henry, Prof. Edwin Ruth van, A.G.O., 270 Madison Avenue, New York, New Y 
enley, Dr. W. Ballentine, President., College of Osteopathic Physicians, 

. Griffen Avenue, Los Angeles, California. ducation, 
Hicks, Mr. Arthur C., Acting Registrar, Western Washington College of Edu 

„ Bellingham, Washington :versit: 
Hill, Prof. J. Lawrence, Jr., Department of Mechanical Engineering, Univer: 

Rochester, Rochester, New York 


or 
Hilliard, Mr. George, Director of Perso 


ВЕС: Education, Kalamazoo, Michig 
Hill, Mr. George E., Direc 
aul, Minnesota 


_ Ў New 
Hilion, = M. Eunice, Dean of Women, Syracuse University, Syracuse 
ork 


ond 
› Western Personnel Service, 30 North Raym 


y of 


т ollege 
nnel and Guidance, Western Michigan C в 
ап е, 
tor, Student Personnel Service, Macalaster College: 


Н n ichigan 
Hoekje, Mr. John E., Registrar, West Michi College, Kalamazoo, Mic 
Holcomb, Miss Leila L., Assistant Coup io ап College, Ka 


еп, 
; Molle ^ Wom 

: Ssistant Counselor, University Residences for 
niversity of Texas, Austin 12, Т 


exas Ames, 
Holmes, Mr. John L., Assistant Director of Personnel, Iowa State Colleges 
Towa Pullman 
Holmes, Dr. Lulu H., Dean of Women, State College of Washington, 
Washington 


Hopkins, Lt. Gg) Everett H., D-V (S), U. 
Naval Training Station, Farragut, Idaho 


Hoppock, Prof, Robert, Department of Education, New York University» 
ork, New York 


a200 
Нш De Wilbur J., Dean of Student Affairs, Kalamazoo College, Kalam 
ichigan 


w 
. Joodro" 
Humphreys, Dr. J. Anthony, Director, Personnel Service, and Registrar, W 
Wilson Junior College, 6850 Stewart Avenue, Chicago, Illinois 


0. ® 
S. №. R., Dept. of Personnel, e 


AMERICAN COLLEGE PERSONNEL ASSOCIATION 309 


Ingalls, Les W., Acting Director, Admissions and Personnel for Men, Middlebury 
College, Middlebury, Vermont 
Johnson, Captain Albert P., С.1.5., Midland Army Air Field, Midland, Texas 
Johnson, Mr. Clyde Sanfred, 202 Administration Building, University of California 
at Los Angeles, 405 Hilgard Avenue, Los Angeles 24, California 
Johnson, Prof. William C., Palmyra College. Diamond, Ohio 
оле. Dr. Edward $., Director Personnel Research, University of Buffalo, Buffalo, 
ew York 
Jones, Miss Katherine J., Executive Secretary, Nursery Training School of Boston, 
355 Marlborough Street, Boston 15, Massachusetts 
Jones, Dr. Lonzo, Dean of the Faculty, Central Missouri State Teachers College, 
Warrensburg, Missouri 
Jox, Mr. Marshall J., Personnel Director, Valparaiso University, Valparaiso, Indiana 
поп, Miss Ann Byrd, Personnel Director for Women, University of Colorado, 
oulder, Colorado 
irkpatrick, Mr. Forrest H., Personnel Administration, Radio Corporation of 
, America, Camden, New Jersey Е n КИ 
Kittrell, ris pano fa Dean, Home Economics Division, Howard University, 
ashington 1, D. C. 
opas, Mr. Toate S., Director, Department of Development, Fenn College, Cleve- 
land, Ohio 
ane, Mrs, Estella Hitchcock, Dean of Students, Frances Shimer College, Mount 
Carroll, Illinois 
Larsen, Dr. Robert P., Outlook Sanatarium, Urbana, Illinois А 
Lauer, Mr. George N., Dean of Men, Central Michigan College of Education, Mt. 
Pleasant, Michigan 
LaBarre, Miss Corrine G., Research Assistant, Western Personnel Service, 30 N. 
Raymond Avenue, Pasadena, California 
ео, Brother L, St. Mary's College, Winona, Minnesota 
ahr, Miss Patricia A., Director, Student Union, University of Nebraska, 14th and 
R., Lincoln, Nebraska . 
€ а Florence K., Appointments Office, Harvard University, Cambridge 38, 
assachusetts 
Leetch, Mr. George N. P., Director, College Placement Service, The Pennsylvania 
State College, State College, Pennsylvania 
Lemon, Mr. Erwin B., Dean of Admissions, Oregon State College, Corvallis, Oregon. 
Lloyd-Jones, Prof. Esther, Associate Director of Personnel, Teachers College, Co- 
lumbia University, New York, New York 
Lyman, Mr. E., Chairman, Board of Personnel Administration, Northwestern Uni- 
versity, 102 Lunt Building, Evanston, Illinois 
MacKenzie Ма. Barbara O'Neil, Personnel Associate, Brooklyn College, Brooklyn, 
ew ork 
Mackenzie, Mr. Donald M., North Central Association of Colleges and Secondary 
Schools, 5835 Kimbark Avenue, Chicago, Illinois 
р, James W., Director of Personnel, Wilmington College, Wilmington, 
lo 
Madsen, Mr. Howard C., Manager Technical Employment and Training, West- 
inghouse Electric and Manufacturing Company, 306 Fourth Avenue, Fast 
M Pittsburgh, Pennsylvania. 
allay, Miss Betty Anne, Director of Appointment Bureau, Manhattanville College 
M of the Sacred Heart, New York, New York 
anson, Dr. Grace E., Adjutant General’s Office, Beekman Tower, Ist Avenue 
M. and 44th Street, New York, New York 
anzer, Mr. Charles W., Faculty Adviser, Washington Square College, New York 
Marke lversity, New York, New York 
arks, Mr. Arlyn, Personnel Secretary, Comptroller’s Office, University of Illinois 
ana, Illinois Ы 
Marshall, Dr. Thomas O., Director of Student Personnel, Colorado State Colle 
Me Сый Collins Colorado 85 
а, trof. William C., Department of Education. iversi а 
olumbia, South Ca » University of South Carolina, 


310 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


diversi anston, 
McCarn, Mrs. Ruth O., Counselor to Women, Northwestern University, Ev 5 
; З 


Illinois е В " 
i rgaret E., 448 E. Mississippi, Liberty, Missouri E Жа 
р иге Testing and Guidance Program, University of Texas, Aus 
s er 

месі е James А., U.S.N.R., Villanova College, Villanova, Pennsylvania 
McCreary, Dr. Otis C., 5151 Alcoa Avenue, Los Angeles, be ea Baltimore, 
McCurley, Miss Mary T., Assistant to the President, Goucher College, 

Maryland : р : А . 
McKinney, Prof. Fred, University of Missouri, Columbia, Missouri 
Miller, President Ernest, Goshen College, Goshen, Indiana uc" f Missouri, 
Mills, Miss Thelma, Director of Student Affairs for Women, University o 

Columbia, Missouri 


; New 
Miner, Mr. Robert J., Assistant Dean of Students, College of the City of 
Morgan, Miss Margaret, Head 


versity, New York, New York 


B а i — lehem, 
organ, Mr. E, Robins, Director of Placement, Lehigh University, Bethle 
Pennsylvania 


; iversity of 
oritz, Mr. R. D., Director, Department of Educational Service, University 
Nebraska, Lincoln, Nebraska 


¢ ington, 

Mueller, Dr. Kate Hevner, Dean of Women, Indiana University, Blooming 

Indiana 15, Ohio 

Nichols, Dean Joseph C., Fenn College, 1983 Е, 24th Street, Cleveland hampton, 

Nield, Mrs. Majory P., Director, Vocational Office, Smith College, Nort 
Massachusetts, 


Nielsen, Mr. Otto Richard, Texas College of Arts and Industries, Kingsville, 
Northrup, Miss Catherine, Assistant Director, Student Personnel, Colora 
College, Fort Collins, Colorado 


E isiana 
Odom, Dr. Charles L., Assistant Professor of Psychology, Southwestern Louis 
Institute, Lafayette, Louisiana 


t А А Texas 
Olson, Miss Dorothy, Director, Texas Union, Uni ity of Texas, Austin 12, ni- 
Onthank, Dr. Karl W., Dean of Personnel ае Johnson Hall, Е 
versity of Oregon, Eugene, Oregon 
rr, Dean Cora I., uskingum College, New Concord, Ohio 
Palmerton, Mr. L R i 


Mines, 
‚ R., Director St d М School of 
Rapid Ci ет foe udent Personnel, South Dakota 


Parks, Mr. Donald S., Personnel Dire 


Partch, Dean C. E., School of Educati 
ersey 


ent 
Pattee, Mr. Howard H., General Secretary, California Association of Independ 
Secondary Schools, 645 W, 10th Street, Claremont, California Cedar 
Paul, Dr. JB, Director, Bureau of Research, Iowa State Teachers College; 

Falls, Towa Я 
Pearl, Miss Penelope M, 


uron, Ann Arbor, М. 
Peck, Miss Margaret, | 


ctor, University of Toledo, Toledo, ad New 
‘on, Rutgers University, New Brunsw! 


„вап: 1027 © 
Fsychological Clinic, University of Michigan, \ 
ichigan Uni- 


» Assistant Dean of Women and Counselor — T 
d » University of Texas, Austin 12, Texa hom! 
enoi, Mrs, C. R Faculty Exchange, University of Oklahoma, Norman, Оша іол» 
niversity of 19 m Assistant Director and Assistant Professor of 
y ( ansas, Guidance Bureau, Lawrence, Kansas ina. 
Plank? Ma William D, University of North Carolina, Chapel Hill, North Car ing 
25 Tot. William B., Department of Mining and Metallurgical Eng! 

Syette College, Easton, Pennsylvania vr etes. AMY 
epartment of Psychology, Box 87, Ohio University; T 
йон. Ту Chester J., Personnel Director, William Jewell College, Liberty, Mi 

ornari, e Luther, rector, Bureau of Appointments and Occupa 
mation, University of Michigan, Ann Arbor, Michigan 


ошї 
1880155 
| In 


AMERICAN COLLEGE PERSONNEL ASSOCIATION 311 


Ramsay, Miss L. Alice, Director, Personnel Bureau, Connecticut College, New 
Hed Min ШКО nd Director, College of St. Elizabeth, Convent Station, 
esd D, Sy Y., Department of Education Library, Chicago University, Chicago, 
Illinois у Я ’ | si 
uo qe рер ымны ix uat. ou 
ui Virginia J., Counselor of Women, Faculty Exchange, University of 
Ries рте ачит Оода Psychologist, Stevenson, Jordon and Harrison, 
Каш DX (a ag ИИИ and Guidance, University of 
bra ac Aq a MM Student Personnel, Texas Christian Uni- 
Ri ey, M Margot ДЫК айр Director, St. Joseph College, Asylum Avenue, 
Rae re fee Director, Louisiana Polytechnic Institute, Ruston, 
Roy, UE A L., 507 Oak Street, S.E., University of Minnesota, Minneapolis, 
Ro à C. H., 103 South Hall, University of Wisconsin, Madison 6, 
Reha tine Jessie L., College of Home Economics, Cornell University, Ithaca, 
Russell, МЕ Mohn Dale, Dean of Students, Division of Social Science, University 
fee ig Cien, ina. Director of Admissions, University of New 
Sage, Mr Nee pata пешие Massachusetts Institute of Technology, 
Sy te ш; We бох Road, Duis Massachusetts о Lindenwood 
За egs, Si ae xe ; WM of Student Personnel, Loyola University, 
Шш G., Vocational Advisement воресвон Veterans Adminis- 
Schonfeld, Dr Tose Ros Si бн Sere New ‘York 35, New York 
caman, Mr. William H., иес Admissions and Bureau о | ppi Lu , 
Sadr Pa gl Sie деш Se t eB 
Shank М bod ke Administrative Associate, American Council on Education, 
She ^ Гоно Plates d decal { Co., Marlboro Inn, Montclair, New 
Sisson, Le Donald E., Classification and Replacement, 270 Madison Avenue, New 
Sistrunk, Mr Henry L, Y. М. C. A, 19 S. LaSalle Street, Chicago, Ilinois _ 
‘nner, Dr. Laila, Dean of Women, Allegheny College, Meadville, Pennsylvania. 
miley, Mr. Е, Kenneth, Director of Admissions, Lehigh University, Bethlehem, 
Smith, "Sen ie tag B., 37354849, Psychological Research Project, Nav. Sec. E., 
Smith, Been Tetas Director of Student Activities, Wayne University, 
Smoke, Me а L., Professor of Psychology, Juniata College, Huntingdon, 
Pennsylvania 


Schne 


312 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


Smutz, Mr. Harold T., Regional Personnel Utilization Consultant, 635 New Federal 
uildi , St. Louis 1, Missouri Eod 2 р ia, 
aside ae erie, Bureau of Guidance and Placement, University of California 
Berkeley, California А si 
St. Clair, Mc Walter F., 25 Waldo Street, Manchester, New Hampshire Bedford 
Stapleton, Miss Mary R., Associate in Personnel Bureau, Brooklyn College, 
Avenue and Avenue H, Brooklyn 10, New York d lege, Cam- 
Stedman, Miss Edith G., Director, Appointment Bureau, Radcliffe College, 
bridge, Massachusetts АА it, 
бадат Miss Jane E., Acting Placement Director, University of Detro! 
MeNichols Road, Detroit 21, Michigan А К Seattle, 
Stevens, Prof. Edwin B., College of Education, University of Washington, Sea 
Washington . E " 
Stogdill, Dr. Emily L., Department of Psychology, Ohio State University, Columbus 
10 se h- 
Spacie, Mr. io G., Mer Counselor, Michigan College of Mining and Tec 
nology, Houghton, Michigan А eges 
Strang, Prof. Ruth, Records and Research, College of Education, Teachers Сойер 
Columbia University, New York, New York Cornell 
Stromberg, Mrs. Jean T., Assistant to the Counselor, One Sage Avenue, 
University, Ithaca, New York si 
Stromberg, Lt. E. L., U. S. Naval Air Station, New Orleans, Louisiana 4 Research 
Stuit, Lt. (jg), Dewey B., USNR, Assistant Officer in Charge of Test ап 
Section, Bureau of Naval Personnel, Washington, D. C. CH Miami 
Super, Capt. Donald E., Psychological Branch, Unit T, AAFR and р 
Beach, Florida 
Sutherland, Mr. Robert L., University of Texas, Austin 12, Texas Christi, 
кош Lt. Reginald L., Н (S) USNR, О.5.№.А.А.5., Rodd Field, Corpus 
exas St. 


Swanson, Prof. Donald E., Director of Student Personnel, Hamline University» 
Paul, Minnesota 7ashington 
Swenson, Miss Hilda G., Social Director, Box 151, College Station, Was 
State College, Pullman, Washington жы 
Tallmadge, Miss Frances M., 328 W. 88th Street, New York 24, New York 
Tarbox, Mr. Sidney, Director of Recruitment of 7th Regiment of О. 
ervice Commission, 12 S. Monroe Street, Hinsdale, Illinois ia 
Tate, Mr. William, Dean of Students, University of Georgia, Athens, Georg? 


9; Civil 


топ, 
Taylor, Mary E., Women’s Residence Center, University of Indiana, Blooming а 
ndiana mb; 
Thisted, Mr. M. N., Dean of Men, Western Illinois State Teachers College, Mace 
inois 


Я ома, 
Thompson, Мг. С. Woody, Director, Office of Student Affairs, University of T 
Towa City, Iowa Pough- 
Thornbury, Miss Zita L., Director, Vocational Bureau, Vassar College, 
keepsie, New York 
Thorpe, Miss Alice L., Appointment Office, Wheaton College, Norton, Massa 
Tibbals, Dean C. A., Illinois Institute of Technology, Chicago, Illinois | , rsity of 
Tibbetts, Mr. Clark, Director of Institute of Human Adjustment, Unive 
. Michigan, Ann Arbor, Michigan Streets Los 
Tiner, President Hugh M., George Pepperdine College, 1121 W. 79th 
Angeles, California 
odd, Mr. J. Edward, 822 E. John Street, Appleton, Wisconsin 


Trabue, Dean M. R., School of Education, Pennsylvania State College, 
Pennsylvania 


Triggs, Dr. Frances O., American Nu 
New York 


Turner, Mrs. Lillian Barbour, Shurtleff College, Alton, Illinois hom? 
Twyman, Mrs, Margaret G., Director, Pirena Service, University of Okla 
Norman, Oklahoma 


" а i is 
Tyler, Miss Leona E., Director of Bureau of Personnel Research, Univ 
Oregon, Eugene, Oregon 


chuset 


State Colles 
New Yorks 


tses Association, 1790 Broadway, 


ity © 


~ 


AMERICAN COLLEGE PERSONNEL ASSOCIATION 313 


Tyson, Lt. Robert, 448 Riverside Drive, New York, New York 
Van Dusen, Ensign Albert C., H-V(S), USNR, Aerial Free Gunnery, Central 
Standardization Committee, NAOTC, Naval Air Station, Jacksonville, Florida 
Van Wagenen, Dr. (Miss) Beulah C., Girl Scouts, Inc., 155 East 44th Street, New 
York, New York 
Voorhees, Dean Edwin C., University of California, Berkeley, California 
Voorhees, Miss Helen M., Director, Appointment Bureau, Mount Holyoke College, 
South Hadley, Massachusetts 
Wallace, Mrs. Frances Gibson, Freshman Adviser, 1520 5th Avenue, Huntingdon, 
West Virginia 
Wallace, Dr. Isabel King, Vocational Counselor for Women, University of Roches- 
ter, Rochester, New York 
Waldrop, Chaplain Robert S., U.S.S. Benevolence, AH-13, с/о Fleet Post Office, 
New York, New York 
Walters, Dr. J. E., Personnel and Industrial Relations, McKinsey and Co., 60 E. 
42nd Street, New York, New York 
Wangeman, Dr. Charles E., Head of Bureau of Placement, Carnegie Institute of 
Technology, Pittsburgh, Pennsylvania 
Wegener, Miss Mary A., Appointments Office, Columbia University, New York, 
ew York 
Weir, Miss Edith, Director, Bureau of Teacher Placement, University of Southern 
California, Los Angeles, California ee 
estover, Mrs. Ada S., Assistant to Dean of Women, University of Nebraska, 
1900 Lyons Street, Lincoln, Nebraska 
estover, Dr. Frederick L., Student Personnel Assistant to the Dean, Brooklyn 
College, Bedford Avenue and Avenue H, Brooklyn, New York 
Whiteman, Mrs. Harriet E. Educational Secretary, New Jersey State Teachers 
. College, Newark, New Jersey 
ightwick, Miss M. Irene, Personnel Director, College of New Rochelle, New 
„ Rochelle, New York : 
йол, Prof. E. R., College of Engineering, University of Washington, Seattle, 
„ Washington 
William, Major Cornelia DeCamp, 2012 S. 5th Street, Arlington, Virginia * 
illiams, Prof. Lewis W., Secretary, Appointment Committee, College of Education, 
,, University of Illinois, Urbana, Illinois 
Williamson, Prof. E. G., Dean of Students, University of Minnesota, Minneapolis, 
innesota 
Wilson, Mr. Eugene S., Amherst College, Amherst, Massachusetts 
inegar, Mrs. R. M., 1303 Milton Way, San Jose, California 
inningham, Miss Jane Louise, Huron Valley Children's Center, 104 Welch Hall, 
Ypsilanti, Michigan 
oellner, Prof. Robert C., Secretary, Board of Vocational Guidance and Placement, 
niversity of Chicago, Chicago, Illinois 
ood, Mr. Austin B., Assistant Professor of Psychology, Brooklyn College, Brook- 
lyn, New York 
Woodhouse, Mrs. Chase Going, Director, Institute of Women's Professional Rela- 
tions, Connecticut College, New London, Connecticut 
Wrenn, Dr. C, Gilbert, 755 North Church Street, Salem, Oregon 
Zerfoss, Prof. K. P., Department of Psychology, George Williams College, 5315 
Drexell Avenue, Chicago, Illinois 


MEASUREMENT ABSTRACTS* 


Achard, F. Н. and Clarke, Florence H. “You Can Measure the Probability of 

Success as a Supervisor." Personnel, ХХІ (1945), 353-373. 

The authors studied 300 successful and indifferent supervisors in a public utility. 
They developed a supervisory rating scale which they found to be quite accurate and 
reliable for the four groups of supervisors studied. Successful supervisors had higher 
Scores in tests of mental ability, personality and breadth of interest, and in ability to 
see and understand quickly (visual perception). The procedure proposed by the 
authors for the selection of supervisors includes: the selection of tests, the selection 
of candidates, sifting out of candidates on the basis of test scores, and final selection 

y executives, adding to ratings based on test scores (a) job knowledge, (b) versa- 
tility and adaptability, (c) mental, physical, and emotional fitness, (d) standing with 
colleagues, and (e) how well they. “click” with prospective superiors. Elizabeth Bell. 


Allen, Mildred. “Relationship Between the Indices of Intelligence Derived from the 
Kuhlmann-Anderson Intelligence Tests for Grade I and the Same Tests for 
Grade IV.” Journal of Educational Psychology, XXXVI (1945), 252-256. 
The Kuhlmann-Anderson Intelligence Tests for Grade I were administered to 

300 school children in the middle of their first year. Near the beginning of their 

fourth grade they were given the same test for that level. Neither M.A., LQ., nor 

с. Av. obtained at Grade IV were adequately predicted from the same scores in 

Grade I. The author believes the low relationship might be due to the differences 

in verbal content between the two tests (the first grade is all non-verbal) and that 

the results question the validity of long-range predictions from a group intelligence 
test at the non-verbal grade level. She concluded that a verbal group intelligence 
test has greater validity for predicting successful achievement in the tool subjects 

When given with an achievement test. This would also assist “in determining 

Whether pupils are working up to their ability." Elizabeth Bell. 


Altus, William D. “Racial and Bi-Lingual Group Differences in Predictability and 
in Mean Aptitude Test Scores in an Army Special Training Center.” Psycho- 
logical Bulletin, XLII (1945), 310-320. 

The dichotomous disposition of four bi-lingual groups, American Indian, 
exican, Filipino, and Chinese, permitted validating testing devices by the bi-serial 
correlation method. The highest average bi-serial correlations between four verbal 
subtests of the Wechsler Mental Ability Scale, Form B, and discharge versus gradua- 
tion, were found in the Negro group, including three in the .50’s. The Indian was 

Next, and the Mexican lowest. The Filipino would have been second in average 

validity but for a low arithmetic subtest correlation. Verbal tests gave lower scores 

to the bi-lingual groups, but did not lose in validity. In mean test scores whites 

Were definitely superior, Negroes were second, and Mexicans third. There was little 

difference between Chinese and Filipino, while the Indian was significantly lowest. 

he data prove nothing as to innate racial differences because of divergent back- 
grounds. Elizabeth Bell. 


Glick, H. N., Flynn, Elizabeth, and Macomber, Lois. “Some Comparisons Between 
the Original and the Revised Stanford-Binet Scales.” Journal of Educational 
Psychology, XXXVI (1945), 177-183. 

This article presents results of two studies conducted at Massachusetts State 

College with respect to certain comparisons between the original and revised Stan- 
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ford-Binet scales. "The most significant finding was that the two scales measured 
practically the same thing. The new scale was found to be more valid as a measure 
of intelligence in testing slightly superior children. Also it was found that the new 
scale tested slightly higher and showed greater dispersion of I.Q.’s than the old. Jane 
Nathler. 


Goldfarb, William. “Note on a Revised Block Design Test as a Measure of Abstract 
Performance.” Journal of Educational Psychology, XXXVI (1945), 247-251. 
The Revised Block Design Test is an adaptation of the Wechsler Block Design 

Test. The adaptation involved two changes. The first was an extension of the 

time limit to five minutes for each card. The second was the introduction of a new 

scoring method whereby the subject received anywhere from zero to three credits per 
picture depending on the degree of accuracy. The test, consisting of seven designs, 
was administered to thirty adolescents with a mean age of 12.3. Each child was rated 
on the Revised Block Design Test, the Wechsler Block Design Test, the Wechsler 

Similarities Test, the Vigotsky Test, and the Weigl Test. All correlations were sig- 

nificant at the one per cent level. The correlation between the block design tests 

was 90. Both block tests offer a good measure of the ability to generalize. he 


Revised Block Design Test has a slightly higher correlation with other criteria 0 
abstract ability. Betty Steele. 


Lewis, W. Drayton. “The Relative Intellectual Achievement of Mentally Gifted 
gad Retarded Children.” Journal of Experimental Education, XIII (1944), 


The purpose of the investigation was to determine the degree of relative achieve 
ment of mentally retarded and mentally gifted children. The subjects consisted 0 
children from four hundred and fifty-five schools from thirty-six states. The Ки hl- 
mann-Anderson and the Unit Scales of Attainment Tests were given to twelve hundred 
subjects for each grade. Reading, geography, arithmetic and language tests were 57 
lected from a battery of eleven tests. The basic assumption was that if either group 
were working up to capacity, the median achievement scores should diverge as muc" 
from the norms as do the median mental ages. The results show that the superior 
group fails to achieve expectations, whereas the retarded group achieves more than 
en Ше author сшде, that relative achievement, kared solely on upper 

, is not significant. i ical 2 
and length of attendance in school. ku. factor MALUM PME 
——— 
London, Ivan D. “Psychol " » el inacy-" 

Frychological Revie, LII ois ar alld ee f 
‚ he purpose of this article is то demonst i % principle 0 
indeterminacy is inapplicable for Psychology. Statistician io eek oleay are ue 
justified in presenting this principle as a sufficient reason for the inability to eliminate 
or evaluate interference. The Principle results from the attempt to extrapolate дио 
the microcosm the same dual concepts of space and time attributed to macrocosmic 
events. The principle applies only to the sub-atomic world. Interference JP 
psychological situations are the results of methodological difficulties inherent in the 
situation. and not because of Heisenberg's Principle. There are two types 0 
phenomena in psychology: (1) Convergent phenomena in which behavior is dett 
mined from the average behavior of its Parts; (2) Divergent phenomena in which 2 
single discontinuous event affects the whole aggregate In he latter phenome? 
the indeterminacy deriving from the unpredictable behavior of a single electron 
deals with isolated and unique electrons which cannot be duplicated. Such 3° 
determinacy cannot be made the basis of a systematic Sij chelogs- Betty Steele. 

———— 
Montagu, M. F. Ashley. "Intelligence of. Northern Negroes and Southern wi 5) 
in thie First World War.” American Journal of Psychology, LVIII dae 


Scores made on the intelli, ini Ne 
! gence tests administered by the U. S. Army t9 

and white draftees during World War I are analyzed vnd discussed. Whites 10 ne 
given state scored higher than Negroes in the same state on all tests and in all st 
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except Kentucky and Ohio, where the Negroes excelled on the Beta tests. However, 
it was also found that the median score made by Negroes in a number of states was 
better than that made by whites in several other states. The conclusion, supported 
by geographical evidence, is drawn that the lower scores of the whites, in the instances 
where they appear, are due to inferior socio-economic conditions, and the generally 
lower scores of the Negroes are similarly explained. Findings indicate that differences 
in performance on the tests between Negroes and whites are due solely to the action 
upon native endowment of differences in socio-economic history. Frances Ё, Smith. 


Munroe, Ruth. “Three Diagnostic Methods Applied to Sally." Journal of Abnormal 

and Social Psychology, XL (1945), 215-227. 

Sally is the fictitious name applied to a normal, fairly well adjusted, Sarah Law- 
rence College-student, one of a group of eleven who were selected for an intensive 
study to determine with what accuracy certain testing procedures could portray the 
wholeness of personality and educative potentialities of the students. The Rorschach 
test, modified for large-scale use, was administered by Munroe, a graphological 
analysis was effected by Lewinson, and Waehmer made an appraisal from spontaneous 
drawings. These findings were augmented by reports from the faculty adviser and 
instructors. The experimenters and teachers worked blindly and independently of 
each other. There was a sufficient degree of conformity between the reports, the 
analysis of the tests, and the accomplishments of the students to suggest that remedial 
methods indicated by the tests might have been of value. Helen Heath, 


Patterson, R. Melcher. “Evaluation of a Prolonged Pre-Academic Program for High 
Grade Mentally Deficient Children in Terms of Subsequent Progress.” Journal 
of Experimental Education, XIII (1944), 86-89. i | 
The Prolonged Pre-Academic Program is an experimental educational unit at 

the Wayne County Training School. On the basis of experimental data it was 

found that mentally deficient. children of LQ. levels 60-79 inclusive, educated on 
the time schedule described, appeared to progress in academic work when it was 
introduced at a rate which enabled them to arrive at approximately the same aver- 
age grade level by 16 years of age as they would have achieved had academic 
instruction started at the earliest possible mental age and been given continuously. 
Jane Nathler, 


Rabin, Albert I. "Psychometric Trends in Senility and Psychoses of the Senium." 

Journal of General Psychology, XXXII (1945), 149-162. 

The Wechsler-Bellevue intelligence scale was administered to 150 individuals 
aged 60-84, admitted to the New Hampshire State Hospital, and 100 of the result- 
ing records were divided into 4 diagnostic groups: Senile, C.A.S., Miscellaneous (not 
including organic), and Non-Psychotic. Analysis of results showed no intra-test 
organizational or structural differences between classes. Large positive deviations 
occurred in the 3 major verbal subtests, while practically all performance subtests 
Showed negative deviations. When scores of each subject were correlated with age, 
only performance subtests showed statistically significant negative correlations, 
though 7's for verbal subtests were also negative. Changes occurring in performance 
and timed subtests appear to be due to age levels rather than differential diagnosis, 
with inflexibility and perseverative tendency as the significant factors in lowered 
Performance. Findings of this investigation fail to agree with Wechsler’s dichotomy 
of tests which “do not hold up with age,” and do not support the statement that 
schizophrenia resembles “premature aging.” Frances E. Smith. 


Ryan, Thomas Arthur. “Merit Rating Criticized.” Personnel Journal, XXIV 

(1945), 6-15. 

The author suggests the use of his Inventory of Personnel as a simpler pro- 
cedure based on “few doubtful assumptions” than the merit rating, until new and 
different methods for evaluating personnel are produced through research, He 
criticizes the graphic rating scale of today as being based on false logic, since it 
Secures ratings on distinct traits, then assigns a total score (“adding apples and 
carts”), thus substituting one trait for another. Also, there is no known way of 
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validating the weights given traits. The halo effect he believes so strong that a finge 
careful rating of a man’s “over-all value ' is probably better. Ratings nne es 
compared or corrected for constant errors with confidence. He also states tha 

have shown their reliability to be generally low. Elizabeth Bell. 


Sargent, Helen. "Projective Methods: Their Origins, Theory, and Application in 
Personality Research." Psychological Bulletin, XLII (1945), 257-293. єй 
The theoretical and historical background of the Projective methods is реса 

and the methods аге discussed with regard to their application, under the classi са: 

tions of materials, functional uses, techniques, and purposes. Fundamental eie 
ments testing the efficacy of the methods have been few, and there is need for Bes 

research bearing on underlying assumptions and predictive value. Experimenta ш 

far conducted demonstrate the basic mechanism of projection and claim some, Cede 

of predictive success for projective methods. Methodological problems P iod 
questions of reliability, validity, and standardization. Evaluation of the me sn 

stresses the importance of an open-minded attitude toward theory тыш E 

innovations in method. Projective methods hold promise in clinical psyc pi 

for a science of diagnosis and treatment; there is considerable evidence that 


t VP i: ^, ef 
are worth thorough exploration. The bibliography contains 274 references. Franc: 
E. Smith. 


» 
Smith, Francis F. “The Use of Previous Record in Estimating College Success. 
Journal of Educational Psychology, XXXVI (1945), 167-175. М chool 
he purpose of the investigation was to predict, on the basis of high-s pres 
records, the number of grade-point scores each freshman would earn in his first ae 
ter in college. The high-school record, based on about eighty examinations evalu: the 
by sixteen instructors at different times, was converted into one figure indicating Тор 
Percentage of recommended grades. А multiple-prediction formula was used М 
included the freshman’s aptitude Percentile, reading percentile, and Englis aa 
percentile. The subjects were 903 freshman students. The standard error of estima a 
rom the multiple-prediction formula was 7,97, The results show a correlation | he 
60 between the high-school record based on a single figure and actual success In 


б * ЛП 9 аг 
first semester in college. However, the score loses its predictive value after а don 
or more. It was found that the best indicator of future scholastic success was 
Previous semester’s record. Betty Steele. 


—— 
б if 
Swift, Joan W. “Reliability of Rorschach Scori tegories with Pre-Schoo 
Children.” Child Diogen? XV (1944) ete йш ildren 
Reliability for Rorschach Scoring categories was tested with pre-school chil 5 К 
as subjects under the following conditions: (1) Test-retest over a 30-day mear 
Reliabilities ranged from +.15 to *.83. ( est-retest over a 14-day interval NIE 
interpolated on the 7th day. Uncorrected r's rang 


1 d 
cted values for scores obtained by combining test ап 
~ 06 
1 hose used in (2). The 75 ranged from d by 
to +.84, all but one being above +.39, while corrected values for scores obtaine 


—+08 to +86. (4) Test-retest o fter 


Swift, Joan W. “Matchings of Teachers’ 
Pre-School Children.” Child Develo 
Investigation was made of the validi 

the Rorschach method when applied t 

consisted of a 250-word personality sketi 

by the children's teachers. Of the thirty 

a result found to be significantly better t 


of 
Description and Rorschach Analyses 
pment, XV (1944), 217-224. py 
ty of the total personality picture g! "terial 
о pre-school children. Validating Tritten 
ch of each of 30 pre-school children, erect 
matchings, 14 were correct and 16 Шр confi- 
han chance at the 1 per cent level О 


. 3.2 
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dence, AII but 3 of the 14 correct matchings were of boys, a result possibly due to 
sex differences in stereotypy of behavior. Frances E. Smith, 


Wells, F. L. “Mental Factors in Adjustment to Higher Education.” Journal of 
Consulting Psychology, IX (1945), 267-286. 

An introduction to the psychometric work of the Grant Study, concerned with 
well-adjusted Harvard undergraduates,” with reference to the “technical problems 
in the measurement of the superior intellect, . . . career choice, as well as educational 
prognosis." It is concluded that the abstraction of the liberal arts college curriculum 
is not the real problem, but the fact that it is offered to many who probably cannot 
assimilate it. "A more highly ideational type of mind than commonly receives it" 
profits most from it. One half of the present college population could be replaced 
from the ranks of those economically not so fortunate. Two broad mental qualities 
are needed for college scholastic success, absorptiveness and creativity. The latter 
is hard to recognize with our present methods of measurement. For this reason, 
Objective tests are believed inadequate for higher education. Eight case studies of 
well-adjusted students are presented. Elizabeth Bell. 


Woodrow, Herbert. "Intelligence and Improvement in School Subjects." Journal 

of Educational Psychology, XXXVI (1945), 155-166. 

Studies were conducted at Litchfield, Ill., and Providence, R. I., to determine 
Whether improvement is synonymous with intelligence and whether there is any 
common factor in improvement. In the Litchfield study, the Metropolitan Achieve- 
ment Battery and the Otis Quick-Scoring Mental Ability Tests were given to fourth-, 
fifth-, and sixth-grade children in May of successive years. Intercorrelations of 
Bains in six school subjects were .12. Factor analyses revealed no gain factor 
common to the six subjects. Three factors were found: 1) an intelligence factor 
determining the I.Q., but not the gain; 2) an arithmetic factor; 3) a reading and 
English factor. In the Providence study the Stanford Achievement Test was used. 

he differences in standard scores for periods of one, two, and three years revealed 
low gain correlations. Three factors were found which were similar to those found 
in the Litchfield study. The conclusion of both studies is that there is no general 
gain factor, and that there is no significant relation between change in score and І.О. 
Bette Steele, 


EVIDENCE ON THE VALIDITY OF THE ARMED 
FORCES INSTITUTE TESTS OF GENERAL 
EDUCATIONAL DEVELOPMENT 
(COLLEGE LEVEL) 


HENRY S. DYER 
Harvard University 


Purpose of the Study 


Tue nature and purposes of the United States Armed Forces 
Institute Tests of General Educational Development are fully 
described in the Examiner’s Manual provided with the tests. 
The battery consists of four tests as follows: 


Test 1. Correctness and Effectiveness of Expression. 

Test 2. Interpretation of Reading Materials in the Social 
Studies. 

Test 3. Interpretation of Reading Materials in the Natu- 
ral Sciences. 

Test 4. Interpretation of Literary Materials. 


All four of the tests are objective in form, the questions being 

wholly of the multiple-choice type. There are two equivalent 

forms of each test, one of which is to be administered exclu- 

sively by the Armed Forces Institute (the Military Form) and 

the other of which is available to colleges generally through the 

American Council on Education (the Civilian Form). 
According to the Examiners Manual: 


“The college level tests are intended for use primarily to 
determine whether or not the individual tested is as capable 
of carrying on advanced college work as the student who has 
taken certain broad introductory or survey courses generally 
offered in the first two years of the liberal arts college, or has 


1 U. S. Armed Forces Institute, Tests of General Educational Develo t A 
ee Level). Examiners Manual. New York: American Council rao 
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reached the same level of general educational development 

as the student who has had such survey courses.”? 

The present study was undertaken to discover whether the 
results of the tests were sufficiently valid for use with veterans 
who might seek admission to Harvard after the war. Specifi- 
cally the answers to three questions have been sought: 


1. Do the test results provide a basis for placing students 
in advanced standing at Harvard? | 

2. Do they provide a sound basis for the selection of candi- 
dates for admission to Harvard? я 

3. Сап they be used іп counseling the veteran on his choice 
of a field of concentration? 


Crawford and Burnham? made a study of Yale freshmen which 
would lead one to expect an affirmative answer to the second 
question. They found that the total of the standard scores ОЛ 
the A.F.I. Tests “correlated as well with Freshman first term 


averages in all courses, as did the average of all College Board 
Achievement Tests." 


Limitations of the Present Study 


With veterans actually seeking admission to colleges in T 
creasing numbers, it is not possible to wait for the appropriate 
amount and kind of data to accumulate on the A.F.I. Tests 
before deciding to use them Ог not to use them. The data 0 
the present study provide no final answers to the questions pro" 
posed, but it is hoped they may furnish some helpful clues use 
ful to college administrative officers, 

The group studied at Harvard was composed of under- 
graduates whose educational careers, for the most part, had not 
been interrupted by military service. Their performance ОЛ 
the tests cannot, therefore, be considered as directly compat” 
ble to that of the college-minded veteran who not only will 


2 O5. cit., p. 3. 


d 
5 A. B. Crawford and P. S. Burnham. “Tri University of the Arm? 
Forces Institute General Educational уи NEG DCN Fducalional and psycho 
logical Measurement, IV (1945), 261-270. ard’ 

* This comparison would have been more enlightening if the College Bo 
Scholasti 


s 
; ‹ Test 
c Aptitude Test scores had been averaged in with the Achievement 
scores. 
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have been away from formal classroom work for some time, but 
who will also have undergone experiences whose effect on his 
learning habits is, at best, difficult to predict. Furthermore, 
the Harvard group took the tests on a voluntary basis, moti- 
vated solely by patriotic considerations and the hope of receiv- 
ing one of a series of monetary prizes. Under these conditions 
it was not expected that the group would constitute a repre- 
sentative sample even of a normal civilian undergraduate popu- 
lation. Its incentives could hardly be considered similar to 
those of the returning veterans. 

There were 114 undergraduates who completed all four tests 
(civilian forms) and on whom there was the essential accessory 
information. The composition of this group is shown in Table 1. 


TABLE 1 
Distribution of the Tested Group According to Class Standing and 


Fields of Concentration* 
Non-Scientific Scientific 
Fields Fields Totals 
Freshmen 29 30 59 
Sophomores . 5 7 10 17 
Juniors 15 12 2 
Seniors . 6 5 11 
"Totals 57 57 114 


* Since the field of concentration is not formally elected until the beginning of 


the sophomore year, the freshmen were assigned to "probable fields" on the basis of 
expressed preference. 


Two indices were available by which the general scholastic 
ability of the tested group could be compared to that of a nor- 
mal prewar class. The first of these was the Verbal Score on 
the College Entrance Board Scholastic Aptitude Test which is 
taken by nearly all students as part of the entrance examina- 
tion. The second was the college rank at the end of the fresh- 
man year. The college rank is reported in seven groups: Group 
1 represents a straight A record, and Group 7 represents an 
unsatisfactory record. Table 2 shows how the tested group 
compares with the Class of 1942 on these two indices. The 
tested group is relatively overweighted with students whose 
interests lie along scientific lines; but this fact should not seri- 
ously impair the results of the study if each of the groups is 
considered separately. There is, however, little question that 
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the tested group on the whole is sufficiently above average in 
scholastic ability to require that an allowance for the difference 
must be made in any general application of the findings. The 
allowance can be made with some confidence because of the fact 
that the range of ability in the tested group, as shown by the 
standard deviations, is not unlike that of the normal ie 
In other words, the tested group provides a reasonably goo 
sampling of the less able as well as of the superior students. 
There is one further important limitation on the present 
study. Any findings related to the usability of the A.F.I. bes 
for placing students in advanced standing will probably not : 
generally applicable to colleges where the program of study an 


TABLE 2 


General Scholastic Ability of the Tested Group Compared with That 
of the Class of 1942 


к —-——=———————=—-——=——==—=—=—== 


llege Rank 
No. of Cases C.E.E.B. Verbal Score ( е Үеаг 
Area of lass of 
Сопсеп- Teste Class Tested Class of Tested С 
tration Group 1942 Group 1942 Group 
с 
M с M o M с M 
7 
Non-Scientific $7 624 605 918 5748 920 377 136 £u 1i 
Scientific .... 57 265 6372 103.6 5513 90.0 3.95 176 4. 6 140 
por оен 114 889 6389 979 5678 92.0 3.86 1.58 4.3 


р : he 
the system of promotion are unlike those at Harvard. Т 
students in this study have not been ex 


“ I- 
posed to so-called “su 
vey” courses. Ordinarily, 


A he 
a student in Harvard College, by "e 
time he completes his sophomore year, takes at least one cou 


in the natural sciences, one in the social sciences, and one in the 
humanities. He is free to select any one of a large number a 
courses in each of these areas. There is thus no guarantee tha 
he will have obtained a broad acquaintance with the mnt’ 
in any given area. The one exception to this rule is that pract 


: P : m- 
cally all freshmen are required to take a course in English co 
position. 


Value of the A.F.I. Tests for Placement in 
Advanced Standing dy 
: n + this stu 
In order to determine whether it is necessary in this ha has 
p to differentiate between freshmen and upperclassmen, ! 


Do ——— VR 
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seemed advisable to look first into the question of whether the 
A.F.I. Tests show any relationship to the number of terms of 
work that a student has completed in college. In other words, 
do the tests measure educational development as it is conceived 
at Harvard? 

The representation from each class was so small and the 
difference in average scholastic ability among the several groups 
was so large that a simple comparison of mean scores from class 
to class would be all but meaningless. "Therefore, in order to 
secure a reasonably adequate answer to the question of rela- 
tionship between the test scores and the amount of academic 
work completed, the method of partial correlations has been 
employed. Although partial correlations in the present in- 
stance will not provide an absolutely rigorous statement of the 
situation, it is believed that they provide a practical approxi- 
mation of the true picture. 

We wish to know the correlations between the A.F.I. Test 
Scores and number of college terms completed when general 
scholastic ability is held constant. For this purpose, it is essen- 
tial that the measure of scholastic ability shall be based on 
evidence obtained before the student entered college. Such a 
measure is found in the Predicted Rank List Standing (PRL), 
an index computed routinely for every applicant to Harvard 
College. The PRL is a composite index based upon the appli- 
cant's secondary-school class rank (hereinafter called School 
Rank) and his scores on the College Entrance Board exami- 
nations. It normally has a correlation of about .65 with Col- 
lege Rank at the close of the freshman year. It is expressed in 
the same terms as the College Rank, that is, a PRL of 1 repre- 
sents highly superior ability and a PRL of 7 represents inferior 
ability. 

Table 3 shows the partial correlations between the various 
A.F.I. Test scores and the number of terms completed in col- 
lege, with initial scholastic ability, as measured by the BRE, 


‚ held constant.’ 


5 The School Rank is converted to a standard score with mean of 85 and d. 
ard deviation of 5. Complete tables of zero-order correlati il тае REANO 
8, 9, and 10 at the end of this article. tons will be found in Tables 
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Using Guilford’s “Table of Significant Values of r, R, and 
t,"* we find that two of the partial correlations in Table 3 can 
be considered statistically significant. For the Scientific Group, 
the partial correlation of .28 between the Social Studies a 
and terms completed is above the five per cent level of confi- 
dence and that between Total Score and terms completed (.24) 
is above the one per cent level. None of the partial correla- 
tions found for the Non-Scientific Group is statistically signi. 
cant. In other words, the present data suggest that, to a sma 


TABLE 3 


d in 
Partial Correlations between A.F.I. Test Scores and Number of Terms Completed $ 
College, with. Initial Ability Held Constant 


-Scienti Scientific Group 
Non eue Group cient 7) 
E r 
Test 1 (Expression) ....... 03 05 
Test 2 (Social Studies) .... 17 28 
Test 3 (Natural Science) .. 12 23 
Test 4 (Literature) ........ 4 01 
Total $соге* 16 24 


* Total Score was obtained by summing the standard scores on the four tests 
extent, the A.F.I. Tests 


3 ent 
measure the educational developm 
of students concentratin 


2. х con- 
5 1n science, but not of students dw 
centrating in social studies and humanities. However, in V 


of the small magnitude of even the significant relationships; o 
findings on the present group indicate that the A.F.I. Tests is 
General Educational Development cannot be used as à jd c 
for placing students in advanced standing in Harvard Col E 
unless a fundamental change were made in the principles 8° le 
erning promotion from one class to another. The value of В 
tests at colleges that offer specific courses in general educat! 
remains to be determined. 
Value of the A.F I. Tests for Selecting Students 
for Admission 


1. 
In view of the small relationships found between the АЕ 
scores and the number of terms completed in college, it W25 


8 Guilford, J. D. Psy, 


/ Book 
chometric Methods. New York: McGraw-Hill 
Company, 1936. Pp. 548-9 


Д 
à 
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that in studying the results further, the factor of number of 
terms completed could be disregarded. When selecting stu- 
dents for college study, one ordinarily tries to find the measures 
or combination of measures that are most predictive of aca- 
demic success, where academic success is itself measured in 
some such terms as average freshman grades, grade-point aver- 
ages, and the like. Mention has been made above of the Pre- 
dicted Rank List (PRL) as the pre-admission index having the 
highest known correlation with College Rank at Harvard. 
Since the PRL is a composite of School Rank and College 
Entrance Board examinations scores, we shall combine the 
A.F.I. Test scores with School Rank in the same manner and 


TABLE 4 


Correlations with College Rank at the Close of the Term in Which the 
AFI. Tests Were Given* 


Non-Scientific Scientific 


Groin Group Total Group 
(N=57) (N=57) (N= 114) 
г г > r 
PRI ЖАНЕТ vs 64 71 67 
Total A.F.I. Score mos 41 52 46 
School Rank 62 63 Р .62 
Total A.F.I.+School Rankt .... 65 66 65 


. * Certain of the correlation coefficients are technically negative, but the negative 
signs have been omitted to avoid confusion in meaning. 

+ The values in this row are multiple R's. They are thus not strictly compar- 
able to the r’s obtained with the three other variables. That is, with a new sample 
one would expect shrinkage in the multiple R’s beyond that to be expected in the 
zero-order r's. 


compare the two composites on the basis of the degree to which 
they predict the College Ranks that were assigned at the close 
of the term in which the A.F.I. Tests were taken. Table 4 
shows how the two composites compare with each other. 

It is apparent from Table 4 that, for the tested group, the 
composite of the A.F.I. Total score and School Rank compares 
favorably with the PRL in the prediction of College Rank. It 
will be observed that the School Rank factor, which is common 
to both of the composites, accounts for a relatively large pro- 
portion of the predictable variance in both cases. One cannot 
say how far this factor will be affected by the interruption in 
education that the returning veteran will have experienced 
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Its predictive power will probably vary from one qni 
another. However, it is scarcely a measure that one would wis 
to discard altogether, since there is no reason to suppose ~ 
the predictive power of the tests—both A.F.I. and College 
Board—will not also suffer in a similar fashion and for many 
of the same reasons. 

Of some interest is the fact that while the PRL appears to 
give a slightly better prediction for the Scientific Group as и 
pared to the Non-Scientific Group, the A.F.L-School-Ran: 
composite seems to predict the academic performance of заа 
groups about equally well. The superiority of the PRL 
science concentrators may well be due to the fact that the Col- 
lege Board examination includes a test of mathematical apti- 
tude which is missing in the A.F.I. series. Crawford and Burn- 
ham in their study at Yale concluded that the College Boar 
Mathematical Aptitude Test is “probably indispensable for 
scientific or engineering majors."* 

In general, the evidence from this portion of the study seems 
to indicate that the A.F.I. Tests are useful as an aid in selecting 
students capable of college work at Harvard. 


Value of the A.F.I. Tests as a Basis for Guidance 

Do the A.F.I. Tests of General Educational Developmen’ 
provide the counselor with tools for advising the veteran wit 
respect to his field of concentration? The data of the presen 
study are inadequate to supply anything but the barest hin 
of an answer to this question. he 

It is probably not wholly unreasonable to assume that | ë 
Non-Scientific Group has, on the average, more ability in t А 
fields of its choice than the Scientific Group, and that the Scien- 
tific Group has, on the average, more ability in scientific SY " 
jects than the Non-Scientific Group. It remains to be Y: 
whether these expected differences in ability are matche 7 
differences in performance on the A.F.I. Tests. Table 5 PT? 
vides evidence on this point. ds 

One finds that the Non-Scientific Group consistently po 
to surpass the Scientific Group in the tests ordinarily associat 


7 Of. cit., p. 268. 
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TABLE 5 
Comparisons of Mean A.F.I. Scores Obtained by Two Groups of Concentrators 
dE o Вене 
roup roup Criti- 
(N=57) (N=57) ар 
Ratiot 
М,* Om, 5.0. М->* Om, S.D.2 Mi- M; 
Test 1 (Written English) 98.8 126 94 96.7 124 93 +21 12: 12 
Test 2 (Social Studies).. 72.2 1.47 11.0 68.5 148 11.1 +37 19 .03 
Test 3 (Natural Science) 61.3 1.96 146 69.1 147 110 -7.8 3.2 .001- 
Test 4 (Literature) .... 66.8 134 100 63.5 129 96 433 18 .04 


* Raw scores were used in the computation of the means and standard devi- 
ations. 

t The standard errors of the differences between the means were computed by 
means of the formula: - 


Gare = мом, + Oy 


with its field of interest, i.e., Written English, Social Studies, 
and Literature. With respect to the Social Studies and Litera- 
ture tests, the differences between the two groups are statisti- 
cally significant, that is, the likelihood is less than five per cent 
that differences of this size would arise as a matter of chance. 
Similarly, the difference between the mean scores on the Science 
test is in the direction one would expect, and this difference is 
of such size that one would expect it to occur as a matter of 
chance less than once in a thousand times. 

The actual magnitude of these differences is, of course, not 
very large compared to the total range of the scores. However, 
if a student were to score relatively high on Tests 1, 2, and 4 
and low on Test 3, one might at least hazard a guess that his 
abilities were more like those of students studying in the broad 
area of social studies and humanities than like those of students 
pursuing the sciences as a major field of interest. 

As further evidence of the value of the A.F.I. Tests tor 
guidance purposes, the degree of correlation of the tests with 
subsequent performance in actual fields of study is required. 
Unfortunately, the present data are not sufficiently numerous 
to provide unbiased criterion measures for each of the several 
fields. Nevertheless, it seems reasonable to suppose that the 
College Rank assigned to each student at the close of the term 
in which he took the A.F.I. Tests provides a rough measure of 
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TABLE 6 


Courses Taken by Two Groups of Concentrators in the Term When 
A.F.I. Tests Were Given 


Non-Scientific Group Scientific Group 
(N=57) (№= 57) 
No. of Average per No. of Average per 
Coursés Student Courses Student 

Natural Science ... 36 6 136 24 
Social Studies .... 89 16 29 5 
Humanities ....... 114 2.0 81 14 
ТӨӨ] е ае 239 42 246 43 


performance іп science for the Scientific Group and a similarly 
rough measure of performance in social studies and humanities 
for the Non-Scientific Group. The basis of this supposition 18 
shown in Table 6. 

On the average, the College Ranks for students in the Non- 
Scientific Group were computed on the basis of 4.2 courses 0 
which 3.6 were taken in the social studies and humanities areas, 
and College Ranks for the Scientific Group were computed Оп 
the basis of 4.3 courses, of which 2.4 were taken in the nature 
science area. Clearly, the College Rank does not provide _ 
“риге” criterion measure, but correlations based upon it may 
be considered approximately indicative of the predictive power 
of the several tests. One would expect to find that the Science 


TABLE 7 
Correlations of Each. of the AFD. Tests with College Rank 


Group Scientific Group sical 
(N=57) (N=57) bo E 
EUM ARR її, AP _ 
n* zn a? 2 2-72 
р T 8 
Test 1 (Written English) .. .38 40 9 1 
Test2 (Social Studies) .... 26 27 $ n 1 йк тү 
Тезї3 (Natural Science) .. 29 .30 .58 66 —.36 19 18 
Test 4 (Literature) ....... LEO LUE Ду, Чыў pai 
n 
* The correlation coefficients are technically negative, but the signs have Hr" 
dropped to avoid confusion i о 


e е rte 
in meaning. Each correlation has been conve she 


" Ж, о 
i the purpose of securing an exact test of the significance W 
differences. (See Fisher, R. A, Statistical Methods for Research Workers» n 
York, 1941. pp. 190 f£.) Since each of the z-values is based on the same num ife" 
cases, the standard error of z is a с 


г onstant: .136. The standard error of the 
ence between each pair of z-values is also a constant: .192. 


Fisher's z-value for the 
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test has a higher correlation with the College Rank of the Scien- 
tific Group than with that of the Non-Scientific Group; and 
that the other three tests have a higher correlation with the 
College Rank of the Non-Scientific Group than with that of the 
Scientific Group. Table 7 gives the correlations for each group. 

From Table 7, it is apparent that our expectations are borne 
out except in the case of the Social Studies Test. Here, for 
some reason, the Scientific Group correlation is higher than that 
of the Non-Scientific Group. It should be noted that this dif- 
ference, large as it is, is nevertheless not statistically significant. 
One reason for the low correlation obtained on this test for the 
Non-Scientific Group may be that the ceiling on the test is not 
sufficiently high for students interested in social studies. In 
other words, the test may not be able to differentiate so well 
among persons who are well read in the field as among persons 
for whom social questions are, on the average, a secondary con- 
cern. The present findings also suggest that the A.F.I. Social 
Studies Test may be useful with the science concentrators as a 
Predictor of general academic performance. The same cannot 
be said for students with a non-scientific turn of mind. 

The one statistically significant difference in Table 7 is that 
between the two correlations involving the Natural Science 
Test. In this instance, the difference is in the direction ex- 
pected, and from the size of the correlation coefficient for the 
Scientific Group, it seems fairly clear that the A.F.I. Science 
Test has a genuine value for guidance purposes. 

As to the remaining two tests—Written English and the 
Interpretation of Literary Materials—the present study has 
produced no findings of any clear significance for prediction 
purposes. In a future study these two tests will be further 
investigated. 

Summary of the F indings 


The findings of this study are tentative and should be re- 
garded with caution. The nature and size of the sample used 
makes impossible any definitive statement regarding the value 
and proper use of the A.F.I. Tests of General Educational De- 
velopment. However, with specific reference to the group 
tested, the results of the study suggest the following: 
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TABLE 8 


Intercorrelations, Means, Standard Deviations 
Non-Scientific Group (N = 57) 


A.F.I. Tests 
School Terms Col- 
Hank PRL Testl Test2 Test3 Test4 Total Com- lege 
pleted Rank 
School Rank .... ... ... 4l 26 25 36 40 28 6 
PRE as. кр 61 59 58 66 73 34 6 
Test 1 «= 420 5 60 Л4 23 38 
Test 2 69 68 86  .33. 26 
Test 3 s 64 87 29 29 
Test 4 vue dem GES S meus эшш cem nemo BR 042 
ПИ амаки e — ee. We See 35 Al 
Terms Completed ... ... ОИ SRAY сле es a 
MBs 912 36 988 722 613 668 2883 24 a 
SD. ауылынын 32 .87 94 110 146 100 222 20 1 


1. Although the A.F.I. Tests show а statistically significant 
relationship with the amount of work completed in Harvard 
College by science concentrators, the magnitude of the relation- 
Ship is so small that the tests do not provide a sound basis for 
placing such students in advanced standing under the present 
System of promotion. 

2. The A.F.I. Tests show no significant relationship with the 
amount of work completed by non-science concentrators in 
Harvard College. These two findings, however, should be 
interpreted in the light of the fact that the tested group was 
not exposed to a curriculum in “general education.” 


TABLE 9 


Intercorrelations, Means, Standard Deviations 
Scientific Group (N= 57 
FI. Tests 


1- 
School Terms Со 
Rank PRL Testl Test2 Test3 Test4 Total Com- lege 


39 и и 37 д 33 5$ 
WEB A вз в ый f 
9 4 25 n 2 5 

9 44 вж д 4 
«+ 47 о 37 3 


че Жул CEP UE 
bie юнен — 37 2 
967 685 63.5 2863 2.1 

93 111 96 1493 20 18 
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TABLE 10 


Intercorrelations, Means, Standard Deviations 
Total Group (N =114) 


A.F.I. Tests 
б] Terms Col- 
Rank PRL Testl Test2 Test3 Test4 Total Com- lege 
am pleted Rank 


S9 20 36 36 49 30 62 


School Rank 
PRI „e 56 63 61 56 74 31 67 


Test 1 ves. gs 46 46 52 .73 23 31 
Test 2 PPS. oss scs л 8 — 0.88 — 337 Д0 
Test 3 mae. ^ mis на 49 50 28 35 
"esti: „аана Pw RE 79 25 42 
ДОШ iuste зж ae » А ee. M6 46 
Tems Compiled csc, кии diio азма зз шы суша meos Д5 
М. ЖИИ нн 912 35 9728 704 652 652 2871 23 37 
BID. асана 33 10 94 112 135 100 208 10 17 


3. The total score of the A.F.I. Tests when used in combi- 
nation with the student's school rank should provide a reason- 
ably good prediction of his subsequent academic success in 
College. 

.4. For the tested group, the A.F.I. Tests appear to measure 
"general aptitude" rather than "general educational develop- 
ment.” 

5. On the average, the score patterns yielded by the A.F.I. 
battery appear to differentiate slightly the students interested 
primarily in the social studies and humanities from those inter- 
ested in the natural sciences. 

6. The A.F.I. Test in the Social Studies may be useful with 
students of scientific bent as a predictor of general academic 
ability and development. 

_ T. The A.F.I. Test in the Natural Sciences provides a useful 
Instrument for predicting college success in the sciences. 


A STUDY OF THE FACTOR STRUCTURE OF 
THIRTEEN PERSONALITY VARIABLES? 


CONSTANCE LOVELL 
University of Southern California 


Introduction 


THE purpose of this study was to make a factor analysis of 
the thirteen variables of personality measured by Guilford’s 
Inventory of Factors STDCR, the Guilford-Martin Inventory 
of Factors GAMIN, and the Guilford-Martin Personnel Inven- 
tory 1. 

These inventories were constructed to measure those per- 
sonality characteristics which previous factor analysis and 
clinical work had indicated as important? The original studies 
showed that the thirteen factors were not completely indepen- 
dent of each other though they were sufficiently separate to 
make individual scores helpful. The present study has in- 
volved factor analysis of the correlations found between them 
for the purpose of determining the clusters into which they fall. 
In other words, it has been designed to investigate the nature 
of more generalized super-factors with which the specific and 
interrelated original factors are loaded. Because it is based on 
intercorrelations, it has involved giving all three inventories to 
a single group, consisting of 200 college students. 


The Inventories 


Е These three inventories provide measures of the following 
actors: 


hel >The writer wishes to express her gratitude to Lt. Col. J. P. Guilford for his 
E pful Suggestions concerning this research. 

Thei TRE Guilford and R. B. Guilford. "Personality Factors S, E, and M, and 

k A easurement,” Journal of Psychology, II (1936), 107-127; “Personality Fac- 

LS, T, and A,” Journal of Abnormal and Social Psychology, XXXIV (1939), 


XEXiV СОЗЫ, ion N and GD,” Journal of Abnormal and Social Psychology, 


. 1. Mosier. « i i i ies,” = 
metrika, AT (1937), 25 Factor Analysis of Certain Neurotic Tendencies,” Psycho 


—287. 
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S— Social Introversion-Extraversion (sociability, tendency to 
seek social contacts and to enjoy the company of others 
as against shyness, tendency to withdraw from social situ- 
ations and to be seclusive). 

T—Thinking Introversion-Extraversion (lack of introspec- 
tiveness and an extravertive orientation of the thinking 
process in contrast to an inclination to meditative think- 
ing, philosophizing, analyzing oneself and others, and an 
introspective disposition). 

D— Depression (freedom from depression and possession of a 
cheerful, optimistic disposition versus a chronically de- 
pressed mood and possession of feelings of unworthiness 
and guilt). | 4 

C—Cycloid disposition (stability of emotional reactions an 
moods and freedom from cycloid tendencies in contrast tO 
strong emotional reactions, fluctuations in moods, and 2 
disposition toward flightiness and instability). m 

R—Rhathymia (a happy-go-lucky or carefree disposition, 
liveliness, and impulsiveness as against an inhibited dis- 

. position and an over-control of the impulses). Р 

G—General activity (tendency to engage in vigorous ove! 


| ЖЕЕ ТШЕ 
action versus a tendency to inertness and a disinclinatio 
for overt activity), 


ul ; " : s- 
A—Ascendance-Submission (social leadership vs. social pa 
siveness). 


M—Masculinity-femininity 
temperamental make-up versus femininity of make-up- 
I—Inferiority feelings (self-confidence and lack of inferiority 
feelings as against lack of confidence, under-evaluation ° 
one’s self, and feelings of inadequacy and inferiority )- d 
N—Nervousness (tendency to be calm, unruffled, and pee 
in contrast to jumpiness, jitteriness, and a tendency t? 
easily distracted, irritated, and annoyed). oS 
O—Objectivity (tendency to view one’s self and surrounding? 
objectively and dispassionately versus a tendency tO ne 
everything personally and subjectively and to be hyP 
sensitive). 


5 -— y le 
Co—Cooperativeness (willingness to accept things and peop 


tes . and 
(masculinity of emotional 


Y 
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as they are and a generally tolerant attitude as against 
overcriticalness of people and things and an intolerant 
attitude). 

Ag—Agreeableness (lack of quarrelsomeness and a lack of 
domineering qualities in contrast to a belligerent, domi- 
neering attitude and an overreadiness to fight over trifles). 


In the construction of the inventories, the following general 
Procedure was used. Items were formulated which appeared 
to be diagnostic of each of the thirteen aspects of personality as 
defined by the previous work done. These were stated in ques- 
tion form, to be answered by “Yes,” “?,” or “No.” Preliminary 
Scoring keys were set up on the basis of the best statistical evi- 
dence at hand. The questions were administered to groups of 
Subjects (e.g., 500 employed individuals in the case of the Per- 
sonnel Inventory I). After the papers were scored with the 
preliminary keys the test of internal consistency was applied 
to every item. Those items which were not sufficiently diag- 
nostic were discarded. For the remaining items scoring weights 
Were assigned in accordance with a method devised by Guil- 
ford. 

Because of the possibility of faking answers to items the 
value of personality inventories has been questioned. Probably 
no satisfactory answer can be given without consideration of 
the purpose for which the inventories are used. Administering 
inventories to a group of prospective employees, who know that 
their chances of work depend on their responses, may be ех- 
pected to yield different results from those obtained in a situ- 
ation where individuals are motivated by the desire to gain 
additional information about their personalities. 

Several investigations have been made in which students 


have been asked to take inventories twice. In one situation 
— 
3 The description of the construction of the inventories has been adapted from 
the discussions found in the manuals for the three inventories. 

Кер: Guilford. “A Simple Scoring Weight for Test Items and Its Reliability,” 
Psychometrika, ТУ (1941), 367-374. 

5 For example, R. С. Bernreuter, “Validity of the Personality Inventory,” Per- 
sonst Journal, XT (1933), 383-386; C. Dowling, “Ability of College Students to 
niluence Scores on the Guilford-Martin Personnel Inventory,” unpublished research 
andy The University of Southern California, 1944; J. A. M. Kimber, “The Insight 
of College Students into the Items of a Personality Test,” unpublished doctor’s disser- 
tation, The University of Southern California, 1945; F. L. Ruch, “A Technique for 
тегесипв Attempts to Fake Performance on the Self-inventory Type of Personality 

est,” Studies in Personality, New York: McGraw-Hill Book Company, 1942. 
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they have been requested to respond according to the way er 
think they are; in the other, according to the way they жа 
like to be, the way they think a well-adjusted individual аг 
respond, or the way they think a good employee would respon . 
Such studies have revealed consistently a difference in scores 
between the two situations. The results show that responses 
can be influenced in a given direction but they also give an 
indication that students do not answer the items, under the 
ordinary procedure of administration, so as to present the ka 
possible picture of themselves. Inasmuch as the present study 
was conducted in a manner similar to the “normal” condition 
in the above investigations, it is probably not unreasonable 0 
assume that a similar attitude toward the inventories was 
present. 3 
Such findings, of importance in relation to the matter f 
faking answers to items, are of course not decisive evidence 9 
the validity of the inventories. More direct information in 
been obtained. In one study; inventory scores for factors 5, д 
D, С, and R were correlated with self-ratings and with an 
by close associates. The reliabilities of the ratings for T and 
were not sufficiently acceptable as criteria against which tO 
validate scores for those factors. For S, D, and R, the correla- 
tions were high enough to indicate that the inventory scores 
were quite valid. 

In another study, the validity of factor M was checked ut 
comparing the distributions of the inventory scores of 50 males 


and 50 females not used in the original standardization grouP- 
Forty-six of the males were ab 


tion of the scor 


of masculinity- 
With the P. 


workers were 


R in 

5 J. P. Guilford and Howard Martin. “Age Differences and Sex Difference yq 

Some Introvertive and Emotional Traits,” Journal of General Psychology, 
(1944), 219-229, 


Ferr н и ded inven- 
* Description of this study is taken from the manual of directions for the 
tory. 
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satisfactory” group on the basis of test results? The inventory 
was taken under conditions in which the subjects were informed 
that their employment status would not depend on the results. 
Of 22 workers judged unsatisfactory by management, 68% 
were detected by the test. Of 26 workers judged satisfactory 
by management, 73% were correctly placed by the test. Other 
studies have yielded results in line with this one? The authors 
of the inventory have pointed out that, in these preliminary 
studies, selection of unsatisfactory individuals was made in 
terms of arbitrary criteria and that more detailed study of the 
jobs in question might have led to the use of different cut-off 
points and greater success. In the manual of directions they 
urge that, for usage of this sort, critical scores be based on 
experience in the specific situation. 

Reliability coefficients for the inventories have been given 
in the manuals. They were computed by dividing the scored 
items for each factor into two random halves, computing Pear- 
Son coefficients of correlation, and then estimating reliability 
coefficients by means of the Spearman-Brown formula. The 
reported reliability coefficients are as follows: S = .90, T = .84, 
D = 94, C -.88, R = .90, G = .89, A -.88, M = .85, I = 91, N=.89, 
О = .83, Ag = .80, Co = .91. 


Procedure 


_ The three personality inventories were administered, accord- 
ing to the directions in the manuals, to four elementary psychol- 
ogy classes at The University of Southern California. Before 
the inventories were given out an appeal for cooperation in 
Securing accurate responses was made. The students were 
informed that they would be given their results individually 
and that the scores they made would have no influence on their 
grades in the course. ‘ 

Two hundred and thirteen subjects completed all three in- 


ventories. They were divided, according to sex and nearest 


age, as follows: 
ss 


аза £ R. M. Dorcus. “A Brief Study of the Humm-Wadsworth Temperament Scale 

2 us Guilford-Martin Personnel Inventory in an Industrial Situation," Journal of 
22124 Psychology, XXVIII (1944), 302-307. 

net Н. С. Магип. “Locating the Troublemaker with the Guilford-Martin Person- 

© ^nventory," Journal of Applied Psychology, XXVIII (1944), 461467. 
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Age: 15 20 25 30 35 40 45 50 
Number of Men: 310120 1 1 0 0 0 
Number of Women: 4 71 3 4 2 1 0 2 


Of these cases those whose nearest age was 15 and those "ir 
nearest age was 35 or above were dropped. This selection le : 
200 cases: 122 men and 78 women. It was made because thos 
at the extremes in age might give atypical results for ol 
students and because the loss of such a small number wou. 
make no appreciable difference as far as statistical significance 
was concerned. 

Raw scores for each of the factors were determined for "E 
subject. These were converted into scaled scores by aan 
the conversion tables in the manuals. The scaled scores R 
scores) were originally set up on the groups used in standar r 
zation to normalize the distributions for the various factors. 

Intercorrelations of each factor with every other factor Eye 
then computed, using the Pearson product-moment metho 7 
These intercorrelations are given in Table 1. Sixty-five 0 
them were significant (at the 5% level), being .140 or — 
Sixty-two of them, .182 or greater, were very significant (at the 
1% level). 

The Thurstone method of factor analysis was used. Cen- 
troid factors were extracted according to the procedure given 


TABLE 1 


Intercorrelations of Factor Scores 


STDC R б A м тг w o А 


22 
S .. 423 638 439 65 379 713 101 591 384 465 м0 237 
T + 645 588.300 —070 197 212 335 "391 405 1% 40 
D ++ SOL 228-00 481 315 740 710 746 325 416 
€ end A08 330. 675 -70 722 25l 
R гр 39 525.039 270 1079 20709 -$69 
G i 458-067 088—231 -1059 - 314 -:300 
үч 256 570 325 460 001 7210 
М э. 326 348 366 006 44g 
1 674 .746 .350 ^29 
N ` 720 470 75 
495 + 1 
5 63 
[4 am 
Co 
— T New 
10 J, P. Guilford. Fundamental Statistics in Psychology and Education. 


York: McGraw-Hill Book Company, 1942. Pp. 104-106. 
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by Guilford." In estimating communality the highest corre- 
lation in each column was used. It was decided to continue 
extraction as long as the range of factor loadings was at least 
~.20 to +.20. This criterion called for the extraction of six 
factors? In the following discussion these are consistently 
called “super-factors” to emphasize their distinction from the 
thirteen original inventory factors. 

With these the communalities for the thirteen factors were 
found by computing the sum of the squares of the super-factor 
loadings for each. Comparison of these with the communali- 
ties estimated at the beginning of the analysis revealed one 
difference of .145, which was considered too large to be toler- 
ated. Accordingly a second set of extractions was made using 
the communalities obtained from the super-factor loadings of 
the first extractions. This time the largest discrepancy be- 
tween the estimated and obtained communalities was .038, 
which was considered well within the limits of toleration. 

TABLE 2 


Centroid Super-factor Loadings and Communalities from Second Extraction 


Super-factor loading В Esti- | 
Factor Obtained rated Discrep- 
commu- commu. ancy 


I п II IV V УІ Bay тшу 


5 761 —477 -.166 -.092 226 .047 896 876 020 
T 560 109 —.521 -.123 -.198 .060 655 ‘617 038 
D  .896 197 -292  .132 1076 -.057 953 969 016 
C  .780 423 -280 260  .089 ~.125 957 `968 011 
R 433 —665 —077 —.182 -.187 .084 71 :698 1013 
G 19 –.740 054 .076 -.116 -.242 643 620 1023 
A 668 —.498 190 195 .081 114 .788 1812 024 
М 351 153 .106 279 -270 227 360 342 018 
I 828 00  .155 «180  .122 -.023 760 1759 001 
М 736 379 076 039 .087  .166 728 743 015 
О  .846 255 188 014 -.062 -.057 822 827 005 
Ag 404 445 217 -472 .188 —.116 .680 657 023 
Co 558 367 349 -.316 -.024 -.068 673 670 003 


——— 


17. P. Guilford. Psychometric Methods. New York: McGraw-Hill Book 
Company, 1936. Pp. 478-488. 

12 Comparison of the standard deviations of the residuals with the standard error 
of the average correlation indicated that not more than three factors should be ex- 
tracted. However, it is considered only a rough test. Coombs’ criterion gave incon- 
sistent results. Tucker’s criterion (revised) indicated that at least seven factors 
should be extracted. Because of the inconsistency of these results it was decided to 
Continue extraction as long as the range of loadings was —.20 to +.20. Beyond that 


Point (with a maximum contribution to communality of less than .04) it did not seem 
advisable to go. 
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The centroid loadings from this analysis, used in the rota- 
tions which followed, are given in Table 2 together with the 
obtained communalities, the estimated communalities, and the 
discrepancies. 

Rotation of the axes was made graphically, according to the 
procedure given by Guilford.* The aim was to minimize the 
size and number of negative entries and to maximize the num- 
ber of vanishing entries.^ Rotation was continued until no 
further improvement according to these criteria could be ob- 
tained. The super-factor loadings and communalities from 
the final rotation are given in Table 3. 


TABLE 3 


Super-factor Loadings and Communalities after Rotation 


Super-factor loading 


Commu- 
Factor nality 
I i m. av v VI 
5 70L O85 — 422. уй мм 390 8966 
T 084 181 — 1625 1053 465 056 6526 
р 2033 438 аз 0 093 162 I7 
С 017 — 480 80 бо _ 00 7550 
R JH М5 —07,- 44 07 ТП 
G 34 091: — 170 —'234 109 08 638 
A 704 383 000 001 — и 50 7825 
M 017 584 — 066 aoo ' xo О 3587 
I О MUSS d iSe 20 0. 2 7596 
N «ЮЗ 542)" 488. — :351 к 558 7 
0 248 601 4771 dis gig © 819 
Ав -4082 4 060: 9330, 0%. по О 6789 
Со МОВИ И 5/1 2505 ез Sai 06 6769 
c SNE VON 
Two negative loadings remained after the final rotation: 
Trait G had a loading of —.170 on Super-factor III and a load 


ing of — 234 on Super-factor IV. In view of the fact that this 


factor had four significant negative correlations with other fac- 
tors, failure to achieve a Positive manifold through rotation 18 
not unreasonable. Negative loadings of С on these super-fac- 


tors, moreover, fit logically into the interpretation given t° 
them.!* 


15 J. P. Guilford, o». cit., 489-491 and 502—507. in 

24 Loadings of +.3 and above or of — -3 and below were considered significant ed 
naming super-factors; those from + 11 to +.29 and from —.11 to —.29 were considere 
as different from zero but too small to be important in identification; and tho 
between +.10 and —.10 were regarded as vanishing. 


35 See data describing Super-factors III and Iv. 


D 
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Interpretation of Results 


Listed below are the loadings of the thirteen factors on 
Super-factor I, in order from highest to lowest: 


Factor Loading Data describing Super-factor I 
G 734 Has tendency to engage in vigorous overt action 
R Jii Happy-go-lucky, lively, impulsive, uninhibited 
S .704 Sociable, has tendency to seek social contacts and 
to enjoy company of others 
A .704 Tends to be social leader 
I 377 Self-confident 
О 248 Objective 
D .233 Cheerful, optimistic 
E 084 May or may not be introspective 
Co .058 . Either tolerant or intolerant 
С .017 May have stable or unstable emotional reactions 
N 003 Either relaxed or nervous 
M -.017 Either masculine or feminine in emotional make-up 
Ag — .082 May or may not be quarrelsome and domineering 


This super-factor has been identified tentatively as a drive- 
restraint variable. Those factors with sizable loadings on it 
appear to have in common an active approach to experience. 
The person with high scores on them tends to engage in vigor- 
ous overt action, to give relatively uninhibited expression to 
impulses, to seek social contacts, and to be a social leader. This 
Super-factor gives the contrast between the individual who 
Pushes out into activity as against the person who has to be 
forced into it. 

. The other positive loadings, though not high enough for use 
їп naming the super-factor, are in agreement with the identifi- 
Cation made. One might expect that drive for response would 
tend to be accompanied by feelings of confidence in and opti- 
mism about reactions made, and that pressure for response 
might prevent an individual from becoming prey to hyper- 
Sensitive reactions. 

.. Moreover, the vanishing loadings seem in accord with the 
identification. It appears logical to think of drive for response 
as being independent of degree of tolerance, emotional stability, 
nervousness, masculinity of make-up, and domineering ten- 
dency. The only loading that is difficult to fit into this picture 
1s the vanishing one on T. However, if one thinks of T in terms 
of the differentiation it makes between extravertive and intro- 
Vertive orientation of the thinking process rather than in terms 
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merely of tendency toward meditation, the vanishing loading 
seems more reasonable. 
Below, in order of size, are the loadings on Super-factor II. 


Factor Loading Data describing Super-factor II 
[0] .601 Objective 
M 584 Has masculine attitudes 
N 542 Calm, unruffled, relaxed 
I 4537 Self-confident 
c 480 Has stable emotional reactions 
D 438 Cheerful, optimistic 
A .383 Is social leader 
Co 357 Tolerant 
T -181 Lacks introspectiveness 
5 .085 May or may not be sociable . 
Ag .060 May or may not be quarrelsome 
R 2045 May or may not be happy-go-lucky 
G -.091 May or may not have tendency to engage 


in vigorous overt action 


This super-factor has been tentatively named a realism vari- 
able. The inventory factors with high loadings on it present 
a good picture of the impersonal and dispassionate realist. He 
views things objectively. He does not Бо to pieces at seeing а 
fish on a hook. He is calm, unruffled, and self-confident (for 
he is objective enough to know that his bad points aren’t his 
whole personality). Also, because of his objective and imper" 
sonal approach, he tends to have stable emotional reactions and 
not to become unduly depressed by passing disappointments: 
One might expect that such an individual might have some 
tendency toward tolerance and leadership, though the rela- 
tively low loadings of these factors are not unreasonable in light 
of the identification. The vanishing loadings present a logic? 
addition to the description of this super-factor. It seems rea- 
sonable to think of this characteristic of realism as being inde 
pendent of degree of sociability, impulsiveness and carefreeness» 
tendency to engage in vigorous overt action, and tendency 
toward quarrelsomeness. 

This super-factor presents a fairly good picture of reported 
sex differences in personality except for the low loading of soci? 
leadership, which is supposed to be more characteristic of me” 
than of women. However, it seems preferable to name фе 
variable in terms of the attitudes and reactions it involves 


rather than to call it simply “masculinity-femininity.” 


С 
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The loadings on Super-factor III, in order, are as follows: 


Factor Loading Data describing Super-factor III 
c .841 Has stable emotional reactions 
D .813 Is cheerful, optimistic 
T .625 Lacks introspectiveness 
N 488 Is calm, unruffled 
о 471 Objective 
1 445 Self-confident 
S 422 Sociable Р 
Ар .330 Lacks quarrelsomeness 
Co 250 Tolerant 
A .090 May or may not be a social leader 
M .066 May ог may not have masculine attitudes 
R -.027 May or may not be carefree 
G -.170 Less active than the average person 


This super-factor has been defined tentatively as an emo- 
tionality variable. At the low extreme on it would be the 
individual characterized by hampering emotional excess. At 
the other extreme (as indicated by the high loadings) would be 
found the individual who is dependably cheerful and opti- 
mistic, free from constant analysis of himself and others, with 
Some tendency to be (1) free of nervous habits, (2) lacking in 
hypersensitivity, (3) self-confident, sociable, and tolerant, and 
(4) lacking in domineering qualities. Such an individual might 
or might not be a leader in social situations, masculine in his 
attitudes, and uninhibited. It is logical to think that he might 
have some tendency to be a “slow mover,” since a person with 
&reat drive for activity would be likely to get into more up- 
Setting situations. However, the negative loading on G is not 
large enough to merit much consideration in the naming of this 
Super-factor. 

For Super-factor IV, the following loadings were found: 


Factor Loading Data describing Super-factor IV 
Ag .748 Lack of quarrelsomeness and domineering qualities 
Co .693 Tolerant 
о 415 Objective 
N 351 Calm, unruffled, relaxed 
I 240 Self-confident 
D .089 May or may not be optimistic 
5 060 May or may not be sociable 
c 059 May or may not have stable emotional reactions 
A .001 May ог тау not be a social leader 
M —.050 May or may not have masculine attitudes 
T —.053 May or may not be introspective 
R —.061 May or may not be carefree 
G - 234 Less active than the average person 
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This super-factor has been identified tentatively as ае 
adaptability variable. The factors with high loading A. E 
to present a picture of the individual whose actions p те. 
enced by the desire for smooth relations with others. de 
not domineer over others or quarrel with them; һе is "s 
of others' beliefs; and he is objective in his interpretations s 
objectivity being necessary for smooth eq icm: tb a 
people). Perhaps because such a person adapts ш self- 
others easily he tends to be calm and relaxed and to 
confident. a Н 

The vanishing loadings fit readily into this picture. ke 
person who is concerned with adapting himself to the кр e 
of others may or may not be (1) cheerful, (2) desirous s Bun 
out of his way to seek social contacts, (3) high in pe ems 
qualities (he might be either a good leader or a good fol nd 
(4) masculine in attitudes, (5) introspective, (6) impu pt 
or (7) stable in mood. Further, one might expect that e 
à person would tend to have rather low pressure for overt - 
ity, since it would make for fewer chances of disagreement 
others. 


Super-factor V had the following loadings: 


Factor Loading Data describing Super-factor V 


T 465 Lacks introspectiveness 

R 441 Carefree, impulsive 

5 245 Sociable 

M .096 May or may not have masculine attitudes 

D .093 Мау ог may not be cheerful 

N 065 ay or may not be calm and relaxed 

A 047 May or may not be social leader 

Co 040 Мау ог тау пої be tolerant 

о .018 ay ог may not be objective 

Ag .010 ay ог may not be domineering : 

G .000 ay or may not be vigorous in action ae 
C = ау or may not have stable emotional reactio 
I =. 


ay or may not be self-confident 
Below are the loadings for Super-factor VI: 


Factor Loading Data describing Super-factor VI 


S -390 Sociable 

A -370 Social leader 
N .258 Calm, relaxed 
I .255 Self-confident 
D 


.162 Cheerful, optimistic 
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[e .090 May or may not have stable emotional reactions 
R .072 May or may not be carefree, lively 
[S .051 May or may not be objective 
M .038 May or may not have masculine attitudes 
Ag .012 May or may not be domineering 
Co -.042 May or may not be tolerant 
T — .056 May or may not be introspective 
G — .088 May or may not have tendency toward vigorous 


overt action ^ 


These two super-factors are too weak to be of any impor- 
tance. Both are merely doublets, accounting for special corre- 
lations (in addition to the influence of the other super-factors) 
between S and A and between T and R. 

One finding of particular interest in this study is the fact 
that no very general super-factor was located which might be 
called *tendency to give the desirable response" or "insight into 
the desirability of the response." Opinion as to just what score 
on these thirteen factors a very well adjusted person should 
possess would vary somewhat from individual to individual. 
However, probably most persons would agree with the authors 
of the inventories that the following scores are desirable: high 
scores on S, D, C, A, I, №, О, Co, and Ag; middle scores on T, R, 
and G; and a score on M depending on sex. If individuals were 
answering the items in terms of their insight into the desira- 
bility of the items, one would expect to find a super-factor in 
which S, D, C, A, I, N, O, Co, and Ag had sizable loadings. 
Nothing approaching this was found. Apparently understand- 
ing of the desirability of certain responses did not have a 
marked influence on results. This finding is in line with the 
material cited early in the report concerning normal and special 
methods of administering inventories. 

The results of this study present interesting suggestions con- 
cerning the structure of personality. On the basis of the find- 
ings one may conceive of personality as consisting of hierarchies 
of habit systems of different degrees of independence and gen- 
erality. The smallest units are the habit systems tapped by 
individual items of the inventories. Many of them are inter- 
correlated. "They fall into clusters because they have in com- 
mon some more general characteristic. These characteristics 
are not only less specific but are on the average more indepen- 
dent of each other. (Such are the thirteen factors measured by 
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these three personality inventories.) They, in turn, are inter- 
correlated to a certain extent. They fall into certain clusters 
because of even more general factors they have in common. 
These super-factors are more separate from each other on the 
average than the less general habit systems. 

More particularly, this study has indicated the following 
four general habit systems: drive, emotionality, realism, and 
social adaptability. In an orthogonal structure such as this, 
a person may stand at any position on the scale for any of these 
four factors. He might, for example, be high in social adapta- 
bility, low in realism, low in emotionality, and average in drive 
A person with a moderately high score on social adaptability 
would tend to score high on both tolerance and agreeableness 
(the more specific habit systems which have this characteristic 
їп common), because the two are positively correlated. How- 
ever, these correlations are low enough so that, in individual 
cases, there might be considerable disparity between standings 
on the two. Therefore separate scores for each are indicated. 
These, of course, are the factor scores from the inventory. 
me more concise and more generalized picture of an indi- 

personality than that provided by the thirteen factor 
scores one would want measures of the four super-factors- 

Equations for predicting such scores from the thirteen C scores 
have been set up using the Doolittle method. In this process 
an arbitrary mean (50) and an arbitrary standard deviation 
(10) for the super-factor scores have been assumed. More- 
over, only those traits with super-factor loadings of .5 or above 
have been used. These prediction equations are as follows: 

1=28.370+1.78G + -682R +.851S +.891A. 

П =36.101 + .8040 +1.202M + .349N +.2991. 
II = 30.894 +2.273C + 745D + 569T 
IV = 33.967 + 1.805Ag +1339Co, — 

, In addition to the above, one further result should be me? 
tioned. The original studies made by Guilford indicated that 
factors D and C were sufficiently independent to warrant seP?^ 
rate measurement. Items were then constructed which 2P^ 
peared to be measuring each. Obviously these items were not 


16 The following multiple correlation coeffici ead Ras = 888; 
Кп.оих:=.729; Кш-срт = .860; Riy-asco =e Were общей Ricans 
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pure measures, for the correlation obtained in this study for the 
scores on the two factors was .90. This indicates that addi- 
tional work on these two sets of items is necessary to bring the 
correlation between scores on the inventory closer to the corre- 
lation of the factors themselves, as found in the original re- 
search. There were a number of other correlations in the 
seventies. These were for factors in separate inventories for 
which no correlations are available. It might be possible to 
lower these somewhat by the removal of impure items. How- 
ever, in view of the general interpretation of the results of this 
study, one would not expect to eliminate all correlation even if 
perfectly pure items for each factor were used. And, as they 
stand now, the correlations are not high enough to enable accu- 
rate prediction of one factor from the other. 

The results as given are, of course, limited by the selection 
of subjects and the procedure used in the study. Generaliza- 
tion of these findings for college students to all individuals is 
not warranted. Further research, set up in similar form, should 
be done with non-college groups as subjects. In addition, the 
findings would be expected to apply only in cases where the 
inventories were given under the conditions of administration 
used in this investigation. One would predict different results, 
for example, if subjects were asked to take the inventories so 
as to indicate how a happy, well-adjusted person would re- 
spond. Factor analysis of scores obtained with such a pro- 
cedure would make an interesting study. 


Summary 


The purpose of this study was to make a factor analysis of 
the thirteen variables of personality measured by Guilford’s 
Inventory of Factors STDCR, the Guilford-Martin Inventory 
of Factors GAMIN, and the Guilford-Martin Personnel Inven- 
tory І. 

The three inventories were administered to two hundred 
college students under standard conditions. The results ob- 
tained in the study are, of course, limited by these selective 
factors. 

For each of the subjects scaled scores were obtained on the 
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following factors: sociability, extravertive orientation of ane 
thinking process, freedom from depression, stability of emo 
tional reactions, carefreeness, general drive, social ascendance, 
masculinity, freedom from inferiority feelings, freedom from 
nervousness, objectivity, lack of quarrelsomeness, and ae 
ance. Intercorrelations between the scores were then compute 
and a factor analysis of the results was made, using the Thur- 


: A ur 
stone method. Six super-factors were obtained. The first fo 
were identified tentatively as: 


I. Drive-restraint (high loadings on general drive, care- 
freeness, sociability, and social ascendance). EL 
П. Realism (high loadings on objectivity, masculinity» 
freedom from nervousness, and freedom from infer! 
ority feelings). mi 
Emotionality (high loadings on stability of emoti g 
reactions, freedom from depression, and extravert! 
orientation of the thinking process). 1- 


IV. Social adaptability (high loadings on lack of quarte 
someness and tolerance), 


ПІ. 


Scores Were set up using the Doolittle method. 


: t 
elation originally found betwee? 
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THE OFFICE OF RADIO RESEARCH: A DIVISION OF 
THE BUREAU OF APPLIED SOCIAL RESEARCH, 
COLUMBIA UNIVERSITY 


MARJORIE FISKE ax» PAUL Е. LAZARSFELD 
Office of Radio Research 

В Tur rapidly increasing development of the radio industry 
In the past two decades has opened up a new and increasingly 
important area of communications research. Radio is accessi- 
ble to more people than any other kind of communication. Its 
effects are therefore of increasing importance to the sociologist, 
the educator and the politician. And since radio in America 
is a privately owned and operated industry, its impact is also 
a matter of importance to networks, advertisers, advertising 
agencies and others concerned with its commercial effective- 
ness. J 

The Office of Radio Research has been functioning for seven 
years, during which it has been concerned with a wide variety 
of radio research problems. In some instances these problems 
could be solved by a relatively simple adaptation of techniques 
used in other fields of research. In others it was necessary to 
develop new techniques to fit the special characteristics of this 
relatively new medium. It will not be possible within the 
Scope of this chapter to enumerate all such adaptations and 
innovations. We shall, rather, touch upon a few which illus- 
trate the interrelationship of radio and other fields of communi- 
cations research.? 


The Objectives of Radio Research 
It is manifestly impossible to study either the content or 


the effect of every radio broadcast that goes on the air. The 
Тт = 


Res 1 This article is a ‘chapter of a book, How to Conduct Consumer and Opinion 
Я кр edited Ьу А. В. Blankenship, which is to be published by Harper and 
эс early in 1946, 

izing m E are indebted to Dr. Bernard Meyers for his assistance in organ- 
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larger body of knowledge, the over-all picture, has to be built 
up over a period of time, segment upon segment, each segment 
representing a study of a particular program or a particular 
group of listeners. Furthermore, studies of particular pro- 
grams or particular groups of listeners may be done from the 
different standpoints of the educator, the politician, the soci- 
ologist, the psychologist and the businessman, each seeking 
answers to different questions. Rarely is one program or one 
group of listeners studied from all of these viewpoints, but each 
study contributes to the sum total of knowledge of radio’s réle 
in our culture. Each contributes new techniques which can be 
used by the other, Altogether, they are gradually being inter- 
woven to form an increasingly important part in the general 
pattern of communications research. 

Some Examples of the Several Approaches to Radio Re- 
search.—The Office of Radio Research has had occasion to do 
research from all of these several standpoints. In studies re- 
ported in Radio and the Printed Page (11) radio was viewed 
educator; reading and listening 
W insights gained as to the rela- 


di as informational media. It has 
studied a particular Program, the Orson Welles “Invasion from 


enabled the psychologist to gain 
logy of panic. 


3 "Swayed By Smith.” A chapter in The Social 
Robert K. Merton, with the assistance of Marjorie Fi 


Psychology of Mass Persuasion. 
published by Harpers early in 1946, 


ske and Alberta Curtis. Т0 
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was subjected to scrutiny. In this study the content of the 
program was analyzed to determine the variety of its appeals 
and the listeners were interviewed at length to determine the 
relative impact of these appeals, and how they came to decide 
to buy a bond from Kate that day. In the course of the analy- 
sis it became apparent that a particular radio entertainer may 
epitomize the social force of radio, reflecting certain trends and 
concepts in our culture, and at the same time reinforcing them. 
Kate, for example, lays great stress on the sacrifices and sacred- 
ness of motherhood. Indeed for many of her listeners she has 
come to typify motherhood: “Only a mother could plead the 
way she does,” even though most of them know she is not a 
mother. In this extolling of motherhood she not only reflects 
one of the basic concepts of our time and our culture, but at the 
same time she reinforces and strengthens it. 

The program sponsor and the advertising agency study 
radio from yet another angle. They are concerned with the 
extent to which their “messages” get across and with the extent 
of acceptance or rejection of their particular programs. Here 
the research focuses on the immediate response-reactions of the 
listeners. An effort is made to determine the extent to which 
a program is liked or disliked and why, and to find out its effect 
on the subsequent attitudes or actions of those who heard it. 
Studies of this kind involve either program research or the test- 
ing of commercials, techniques which we shall consider shortly. 

Viewing radio as a whole it is apparent that its content is 
the product of contemporary culture and customs. Its content 
reflects this culture because the people responsible for it are in 
turn the products of it. Analyses of the content of radio broad- 
casts, therefore, provide the sociologist with a better under- 
standing of society. On the other hand, this content has an 
impact on millions of radio listeners, and by studying the effects 
of radio on listeners’ habits and attitudes the sociologist also 
gains insight into the way in which it is changing, modifying or 
reinforcing the cultural pattern. 


The Techniques of Radio Research 


‚ Surveys of General Listening Habits—The most general 
kind of listener survey involves a simple count of how many 
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people listen to a given program. Such counts, known as pro- 
gram ratings, are based on careful samplings of the population 
and are made systematically by a number of research organi- 
zations. Program ratings and fluctuations are thus made avail- 
able regularly to private clients.* The Office of Radio Research, 
however, while it does make use of such ratings in some of its 
more quantitative studies, e.g., “The Social Stratification of the 
Radio Audience” by Hugh Beville (2), confines itself largely 
to studies of a more detailed psychological nature. 

Surveys of general listening habits may be made for several 
reasons. One may wish to compare the influence of the various 
media of communication on a given area of behavior. How, for 
example, does the influence of radio compare with the influence 
of newspapers and magazines on voting behavior? (13). Or 
one may want to measure changes in listening habits resulting 
from program changes, or to gain insight into the róle of radio 
among certain groups of people. In this case one must also 
survey general listening habits over a period of time. Whatever 
the purpose of such general surveys, the procedure is the same: 
something akin to a “listening diary” must be procured from 
a representative sample of the population one wants to study- 

Several studies of this kind, both commercial and non-com- 
mercial, have been made at the Office of Radio Research. 
good example of this is the question of listening in the daytime 
If the whole listening pattern of women were taken into con- 
sideration they would fall into three equally large groups. Day- 
time Listeners include those who listen to serials and those who 
listen to other programs. Those who are at home in the morn- 
ing and could listen but do not comprise the Non-Listeners. 
Thus radio actually does not reach a third of the available 
morning audience and, at the same time has to cater to tW 
rather different kinds of audiences, How to reconcile the in- 
terests of these two divergent sectors of the audience is а Prob" 
lem which leads to a large number of interesting and still partly 
unsolved research problems. 

Another survey of general listening habits was concerned 
with the question of who listens to small local stations. This 


4 See Chapters XI, XII, XIII, and XIV. 
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involved interviewing a cross section of the radio audience in a 
given locality about their radio listenership over a period of 
time, and comparing different age, sex and socio-economic 
groups in respect to station preferences (16). It developed 
that people on the lower socio-economic strata tend to listen to 
such stations more than do those who are better educated and 
better situated financially. 

A study of a somewhat different nature recently completed 
by the Office also falls into this category. Here the problem 
was to determine (a) the degree of satisfaction with current 
radio offerings, (b) attitudes toward commercial advertising on 
the radio, and (c) general receptivity toward a proposed new 
plan by which the listener would subscribe to a service which 
would provide him with three types of radio programs without 
any advertising. 

Still another kind of general listenership survey is designed 
to determine the réle of radio in the lives of particular groups— 
children, for example, or housewives, or certain socio-economic 
groups. Such a study usually involves careful and detailed case 
histories of representatives of the group under study. This 
kind of investigation is well exemplified in two Office of Radio 
Research studies, “Listeners Appraise a College Station” (4), 
and “Radio Comes to the Farmer” (19). In the latter, it was 
Possible, by use of the detailed interview method, to determine 
the extent to which the acquisition of a radio changed the habit 
and thought patterns of a group of Iowa farm households. 

A similar type of survey is indicated when a sponsor or а 
group of sponsors wants to measure the impact of his radio ap- 
Peals or to compare it with appeals in other media (12). Tt was 
found possible in one study, for example, to gauge both the 
actual results of radio advertising and to get some idea of its 
potentialities (6). The investigators went into a representa- 
tive, moderate-sized community and interviewed several hun- 
dred housewives at great length about their listening habits, 
their awareness of retail merchandising over the air, their vary- 
ing degrees of receptiveness toward the various kinds of retail 
advertising and the extent to which such attitudes influence 
their buying habits. Among other things this study indicated 
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that there are certain kinds of programs which are better 
adapted to the selling of retail merchandise than others, and 
that certain kinds of merchandise lend themselves better than 
others to advertising over the air. 

In the course of such studies it has become clear that non- 
listeners are also an important factor in radio research, both 
from the standpoint of particular programs (17) and from the 
standpoint of non-listening in general. If we know why cer- 
tain people do not listen to a given type of program and how 
many people do not listen for these reasons, we can plan pro- 
gram changes which may not only improve content level but 
at the same time increase the total amount of listenership- 
Similarly, if an extensive survey were made of people who 
seldom or rarely listen to the radio at all, we could round out 
our picture of radio as a cultural expression and a,cultural tool. 

The Nature of Program Research.—Such broad audience 
surveys as those outlined above cannot possibly encompass the 
more specific problems of listener likes and dislikes, listener 
gratifications or the extent to which listener attitudes аге 
changed or modified by radio listening. Therefore, the investi" 
gator finds it increasingly necessary to study particular pro- 
grams or series of particular programs. In doing so, however, 
he comes face to face with research problems which are not 
especially peculiar to radio but which have their counterparts 
elsewhere in the communications field. How do you measure 
listener reaction to a Program? What specifically does a lis- 
tener mean when he says he liked or disliked a certain program? 
How can one measure the "effectiveness" of a given informa- 
tional program? How determine the cumulative effect of 2 
series of programs? 

There are three different ways of learning what a program 
means to people: by subjecting the program to a content analy- 
sis, by making a differential analysis of the personal character 
istics of the groups that listen to the program, or by asking 
people directly what the program means to them. Wherever 
possible all three methods should be used simultaneously. 

Content analysis of radio material involves essentially the 
same techniques as are used in the analysis of printed materials; 


| 


THE OFFICE OF RADIO RESEARCH 357 


and are usually based on scripts or transcripts of the broadcasts 
(unless one is studying all programs for the occurrence of a 
certain type of a content in which case one has to resort to 
“monitoring”). From analysis the investigator is able to list 
most of the affective factors of a broadcast. Thus the content 
analyst, after listening to a few instalments of a daytime serial 
script, may learn that it stresses an individualistic, competitive 
type of social relationship, that the surgeon hero wants to be 
a great man and stand high in a prestige hierarchy, that his 
interest is in himself and not in humanity. Ог he may discover 
that the negro character depicted in the series is a servant whose 
chief characteristic is doglike devotion to his master with little 
or no portrayal of any individual thoughts, feelings or individu- 
ality of his own. In another study a content analysis of a Kate 
Smith script reveals that she sometimes uses the word America 
or American as many as seven times in five minutes, thus 
building up an associative complex which contributes to her 
reputation as a patriot. RA 
But, as we have already suggested, content analysis is im- 
portant largely as the first step in the study of any particular 
program or series of programs. In subjecting the script or 
scripts to a preliminary content analysis, the investigator accom- 
Plishes two objectives. He is able to distribute his questions on 
the various components of the broadcast with some regard. to 
their frequency and importance. Secondly, a thorough-going 
content analysis permits certain inferences about what the lis- 
teners may get out of the content, or at least will give the 
Investigator some idea of what not to look for. Because it pro- 
vides both balance and perspective, content analysis is usually 
the first step in program research, whether it be an investigation 
of one program or a series of programs. 
_ The second way to find out what a program means to people 
is to determine what sex, age, and social groups listen to it. 
Much is known about the psychological differences among vari- 
ous strata of the population, and if the program is listened to 
by some of this group more than by others, the nature of its 
appeal can be more readily understood. If, for example, the 
audience of one of two comedians is more highly educated than 
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the audience of another, then it can safely be assumed that the 
first offers a more sophisticated kind of humor. The character- 
istics which are to be isolated will of course vary with the prob- 
lem at hand. In a study of the audience for a child guidance 
program, for example, whether or not the listener has um id 
was a pertinent factor. It was found here that quite a num er 
of childless women were found among the regular listeners, 
hence the conclusion that the practical advice offered is not the 
only appeal in this program. Some women, regretting their 
lack of children, might derive a vicarious satisfaction 0 
hearing child problems discussed, while for others the broad- 
casts might have general educational value. е 

If more general listening habit surveys included detaile 
information about the listener, such as reading habits, leisure 
time activities, community participation and so on, such mate- 
rial would become a useful tool for the further analysis of what 
certain kinds of programs mean to listeners. 

One of the major problems of program research is how to £et 
the respondent to indicate what, in the program or series of pro- 
grams under study, is responsible for his reactions to it. y 

` means little or nothing to the program planner, for instance, ! 
he is told that 70% of the respondents studied liked a program 
very much, 20% liked it moderately well and 10% did not like 
itatall. It does not tell him what could be done to improve the 
attitudes of the other 30% or whether changing it to meet n 
taste would not at the same time antagonize the 70% who like 
it in its original form. After considerable experimentation, how- 
ever, the Office of Radio Research has developed a technique 
which seems to contribute to the solution of this basic problem- 
This technique involves the adaptation of the polygraph fre- 
quently used in experimental psychology, and is known as the 
Lazarsfeld-Stanton Program Analyzer. 

The Program Analyzer is an apparatus which enables ? 
group of respondents to record their reactions to a radio p10- 
gram, as they listen to it, by pressing red (“dislike”) and E 
(“like”) buttons, or by not pressing buttons, which signifie 


indifference to what is being heard. The push buttons are COP ` 


nected with a pen which moves along a roll of tape synchronize 
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with the radio program, thus making a permanent record of the 
reactions of the group (8, 15). 

Such a record alone, while interesting as a picture of the high 
and low points of the program as far as a given group of lis- 
teners is concerned, is comparatively meaningless from the 
standpoint of program improvement. The important thing for 
the program planner or the educator to know is what there was 
about a particular part of the program that the listener found 
dull or interesting and why in terms of the listener’s own back- 
ground and experience. The Program Analyzer Technique is 
therefore nearly always combined with a focused interview" in 
which the trained investigator, using the Program Analyzer 
graph for reference, is able to determine just what it was about 
the program that caused the reactions indicated on it, and just 
what these reactions mean in the experience of the listener. 
Every research man who has tried to determine the “why” of 
reactions to a particular experience will recognize the advan- 
tage of this method. It gives him a picture of reactions which 
occurred simultaneously with the experience, and obviates a 
frequent difficulty in retrospective interviewing, to wit that the 
respondents often fail to remember how they felt in the earlier 
Parts of the experience. Like many techniques developed in 
One field of communications, this one is useful in others as well 
and has been used successfully by the Bureau of Applied Social 
Research in testing reactions to motion pictures. 

The Program Analyzer, of course, can be used to study 
Teactions to any kind of program. It has been found useful in 
determining the effectiveness of educational broadcasts, in 
analyzing the appeal of entertainment programs and measuring 
the impact of commercial announcements. The usual proce- 


ure is to interview 10 or 20 groups of people (10 to 15 ina 
ee 
. 5 The focused interview is a term applied to the technique of determining reac- 
tions to a particular communication or experience (a motion picture, a radio program, 
Printed materia] and so on), known to the investigator, as distinguished from the 
more diffuse type of interview which is required when studying listening habits or 
attitudes which may be the result of several different experiences which are usually 
enknown to the investigator. The focused interview is a rather complicated pro- 
SS те and the O.R.R. is now in the process of codifying the results of its experience 
Um this technique with various media of communication. The results of this sys- 
aematization have been summarized by Robert K. Merton and Patricia Kendall, in 
in puse "The Focussed Interview" to appear in the American Journal of Sociology 

the spring of 1946, 
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group), carefully selected to be representative of the audience 
the program is designed to reach. They first listen to the pro- 
gram, recording their reactions with the Program Analyzer 
push buttons, and are then questioned by a highly trained 
interviewer with all remarks recorded by a stenotypist. Their 
comments are then analyzed in conjunction with Program 
Analyzer graphs, and the investigator can thus determine what 
the effective components of the broadcast were and can make 
recommendations as to which parts of the program should be 
taken out, changed, or eliminated for more satisfactory results. 
When a radio investigator wants to probe deeper, to deter- 
mine the gratifications of certain segments of the radio audi- 
ence, interviews of a more elaborate kind are in order. Such 
studies usually involve two steps: detailed and exploratory case 
studies, followed by less detailed interviews with a larger sam- 
ple, for statistical verification of the hypotheses developed from 
the qualitative data. This combination of qualitative an 
quantitative research has two advantages. On the one hand, 
the first step enables the investigator to gain rich psycholog!- 
cal insights which permit him to cover the wide range of possible 
responses in the statistical survey. On the other hand, the 
qualitative material enables him to understand, clarify ап 
illustrate the quantitative data more fully. The combination 
of the two types of research has proven so fruitful that it has 


become an established procedure in many of the studies under- 
taken by the Office. Perhaps the best w 


is by way of a concrete example. 


An Example of Program Research.—The problem was t° 
determine the gratifications of the millions of women who liste? 
to the serial stories broadcast throughout the day by the major 
networks. As a first step, 100 women from various age АП 
Socio-economic groups were interviewed intensively (9). An- 
alysis of their reports about their listening experiences and the 
satisfactions they derive from it indicated that there are three 
major types of gratification in listening to daytime ѕепіа $ 
Some listeners enjoyed them primarily as a kind of emotion? 
release. Burdened with their own problems, they claime 1t 
“made them feel better to know that other people hav? 


à : e 
ay to illustrate its valu 
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troubles, too.” A second and more obvious form of enjoyment 
of the serials comes from the vicarious experiences they supply. 
A third gratification was entirely unanticipated by the investi- 
gator and constitutes a good illustration of the value of this 
kind of intensive interviewing. Many women listened to serials 
because they provide standards of value and judgment and help 
them to solve everyday problems. They learn things from 
these stories which they use later in solving their own problems: 
“Bess Johnson shows you how to handle children. She handles 
all ages. Most mothers slap their children. She deprives them 
of something. That is better. I use what she does with my 
own children.” Or they provide comfortable philosophy for 
use with one’s self or others: “When Clifford’s wife died in 
childbirth the advice Paul gave him I used for my nephew when 
his wife died.” 

In this way, by providing such “leads,” the intensive inter- 
views opened up the areas for investigation on a more quanti- 
tative basis, Later, when 2,500 listeners were interviewed (10), 
41% claimed to have been helped by daytime serials, thus 
giving statistical validity to a gratification which might have 
been overlooked altogether had the intensive interviews not 
been made. With the larger sample it became possible to make 
Cross-tabulations which showed what kind of women found the 
Serials helpful in this way. Thus, for example, it developed that 
the less formal education a woman has the more help she derives 
from the serials. The quantitative material also made it possi- 
ble to analyze the nature of this help, and it developed that 
listeners find these programs useful in several ways: getting 
along with people, helping people with their personal problems, 
learning how to handle themselves in particular situations, 
learning how to accept misfortune with a smile, and so on. 

. Analysis of listener reaction combined with content analy- 
sis (1, 7, 14) of the scripts themselves then led to certain 
inferences about the róle of such programs in our culture. It 
was found, for example, that these so-called true life stories do 
not deal with basic social or economic problems. They do not 
show a woman how she can improve her economic status nor 
do they give her a better understanding of the current problems 
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of our time—e.g., minority groups, etc. They tend, rather, to 
imbue the listeners with a fatalistic philosophy of life: this is 
how it is, we aren’t as badly off as we might be. They help the 
listener to accept her fate by universalizing it—e.g., “husbands 
never understand their wives.” Thirdly, they encourage the 
listener to live life through ready-made formulas for behavior 
rather than helping her to develop a critical sense which will 
enable her to determine what is good or bad for her in a particu- 
lar situation. 

The field of program research, however, has been by no 
means completely explored. Little, for example, is know? 
about the maximum potential of radio from an educational an 
cultural standpoint. We know, to be sure, that by and large 
the programs that are known and promoted as “educational’ 
reach a relatively small proportion of the radio audience, chiefly 
those who would make a point of acquiring the same informa- 
tion from another medium if it were not available to them ove! 
the air. It is known that such programs will not reach eve? 
these relatively few listeners unless organized efforts are made 
to build an audience (14). But what about the utilization 0 
such already accepted programs as the daytime serials as 2 
means of raising, rather than catering to, the cultural level o 
the average listener? The sponsor feels he would thereby 105€ 
Some of his audience. But the fact remains that few have trie 
to improve them and there is as yet no proof that the sponsors 
are right or wrong. 

Effect Studies —Another more specialized form of radio 16 
search pertains to the effectiveness of one section or element 0 
a program. The commercial sponsor may want to determine 
the effectiveness of his commercial announcement. He may 
want to compare the effectiveness of two or more different 
presentations. The program planner may want to determine 
the extent to which his program depends upon the popularity 
of any single feature in it. He may want to compare commen” 
tators or announcers to determine which one is most acceP™ 
able to the greatest number of listeners. In all these cases the 
research procedure, as in most stimulus-response 500 ез; 
involves holding all factors constant except the one Un ён 
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study. Thus, to determine the relative appeal of two com- 
mercials, matched groups of respondents (or sometimes the 
same respondents) will listen to two broadcasts which are alike 
in all respects except the commercial. If there are no extrane- 
ous factors involved all differences in reaction to the two will 
be the result of differences in the appeal of the two commercials. 

If the sponsor does not have two or more specific com- 
mercials which he wants to compare, but wants, rather, to 
determine the effectiveness of a particular one, the problem is 
somewhat different. In the first place, he must decide whether 
he wants to measure the effectiveness of the commercial in 
terms of the number of sales of his product which it induces or 
is likely to induce, or whether he is concerned only with the 
extent to which the commercial is liked or disliked. (The rela- 
tionship between liking a commercial or any other kind of per- 
Suasive appeal and being induced to act as a result of it is, 
Incidentally, a problem which needs much further exploration. 
Studies done to date indicate that one may dislike a commercial 
intensely" spot" announcements, for example, or singing com- 
mercials—and still be influenced by them.) If the investigator 
15 primarily interested in sales effects rather than in what ele- 
ments of the commercial make it effective, a controlled check 
is commonly used. A section of the population is “exposed” 
to the appeal and sales figures for the product in that area are 
checked against those in a comparable area where the popula- 
tion was not so exposed. An alternative to this procedure in- 
volves interviewing buyers of the product to determine how 
they came to buy it. Most advertisers, however, seem to oper- 
ate on the theory that there is a connection between liking a 
Commercial and buying the product which it extols. Conse- 
quently they are interested in research which will determine 
the degree of acceptance or rejection of the commercial an- 
Nouncement itself, This problem involves quite different 
techniques. 

If the advertiser is concerned only with the interest aroused 
by the commercial in a given program context, the Program 
Analyzer technique is in order. The graph will show clearly 
the relative position of the commercial within the framework 
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of reactions to the program as a whole. From the focused й 
view which follows Һе сап then learn much about wae a 
liked or disliked about the commercial and what in terms o 
listeners’ own experiences caused the favorable or unfavora : 
reaction. This technique is also useful in determining the effec 
tiveness of commercials placed at various stages of the pro- 
gram—e.g., is the commercial placed at the beginning, end 
middle of the program more effective? Should it follow a pea f 
of interest in the program to capitalize on the high degree o 
attention at that point, or would such an approach cause a e 
down” on the part of the audience which might boomerang — 
resentment that "something is being put over" on the манын 

Another technique for studying commercial апаран. a 
has been found especially useful in testing reactions to а touc a 
subjects, very personal products or in testing institutional a h 
vertising. This involves an intensive “depth” interview bie 
is made immediately after the subject has read or heard t 
advertisement, and is equally useful for printed advertisements: 
Here the interview is customarily of an associative даш 
What words or ideas are taboo? What words cause unpleasan 
associations which might in turn result in an unfavorable att" 
tude toward the product or sponsor? This technique, inciden- 
tally, has suggested the interesting possibility that pp 
matters can be discussed in print which are not acceptable aver 
the air and that contrariwise, some approaches are more effec 
tive orally than in print. 

By and large, however, it has been found that the best ei 
to make people articulate about commercial announcemen 
a subject which often leaves them lethargic at best, is a 
them compare two or more. Most people are not sufficient 7 
interested in such matters to become very talkative about the 
reactions, and the necessity of making a choice between ы и 
more often provides the necessary impetus to self-examinat! 
as to why they selected one or the other. xe 

Another type of program research problem to which we ee 
already alluded involves the program series. To wea s 
just one of a series of educational or entertainment or dram 
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programs would not be a valid test of the effectiveness of the 
series, for reactions to a given program may be partly predicated 
on remembrance of what went before and expectation of what 
istocome. Then, too, if the investigator is interested in changes 
of attitudes as a result of a program series he will get little from 
merely testing one program. The technique which has been 
developed to solve this problem is called the “panel,” which, 
reduced to its barest essentials, involves the selection of a group 
of people who agree to listen to a series of broadcasts and then 
report their reactions to the various programs. They may agree 
to come to a given place in a group and-participate in a group 
interview using the Program Analyzer, or they may agree to 
record their reactions to various programs on formal question- 
naires (the latter, of course, is in order when a nationwide 
sampling is desired). Thus a virtually constant and identical 
group is made available for the examination of a series of pro- 
grams, and detailed comparisons of one program with another 
in a series can be made. Obviously, one can also procure other 
information from such a group—reading habits, program prefer- 
ences, movie attendance, and so on, which is helpful in evalu- 
ating variations in listener reactions. 

Special Characteristics of Radio.—There are at least six 
Characteristics of radio which distinguish it from other media. 
Researchers in the Office have had to take these into considera- 
tion when studying its effectiveness from any particular stand- 
Point. Each of these qualities has both positive and negative 
aspects from the standpoint of effective communication. 

Perhaps most significant of these characteristics is radio's 
accessibility. Nearly every person in the United States has 
access to a radio. There are few geographic or economic bar- 
riers to its use once the initial investment has been made. In a 
sense, then, radio is more readily available than the other mass 
media, for each magazine, newspaper and motion picture must 
be purchased separately. But in another sense radio is less 
accessible than these other media. Once one has bought a 
newspaper or magazine one can keep it. It may be read at any 
time and an interruption or a lack of comprehension of a pas- 
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sage are not serious matters, for it can always be re-read. But 
a radio program is as ephemeral as time itself. If the telephone 
or doorbell rings just when John Kieran is about to answer а 
baseball question or just when the mystery is about to be solved 
one cannot set back the needle to pick up what has been lost. 
Motion pictures are also ephemeral in this sense, but the си- 
cumstances under which they are seen tend to offset this factor: 
one is not likely to be interrupted in a theater, and if a movie- 
goer so desires, he can always sit through a second showing. 

Another special characteristic of radio is that it relies on 
auditory perception. -The human voice is a more personal, 
direct and potentially more stimulating means of communica- 
tion than the printed word, but this does not necessarily mean 
that radio is a more effective means of transmitting all kinds of 
communication to all kinds of persons. Studies done by the 
O.R.R., for example, indicate that people in the higher social 
and economic brackets more often prefer to read factual infor- 
mation rather than to hear it. 

A third characteristic of radio is in part the outgrowth of 
the first two. Its accessibility, combined with its reliance ОЛ 
auditory perception, enables people to listen while carrying ОП 
a variety of other activities which do not necessarily interfere 
with their perception. But at the same time this quality of 
non-interference leaves the radio program liable to a low degree 
of attention. The listener may become so conditioned to it that 
he no longer hears it with any degree of acuity. This poses ® 
problem for the investigator and necessitates a thorough prob- 
ing of the seemingly factual statement: “Yes, I listened to that 
program” to determine the 
behind it. 

A fourth characteristic of radio is that it continues in time 
This means that a series of programs may become part of the 
daily or weekly habit patterns of the listeners, that cumulat? 
effects can be built up over long or short periods. But it also 
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means that it is liable to surfeit. It may be true that “if you 
hear a thing often enough you will come to believe it,” but it is 
probably equally true that if you hear a thing too often you 
may not pay any attention to it at all after a time. Just where 
repetition ceases to be effective, just when saturation points are 
reached, is still a problem which has to be faced anew for each 
kind of program or message. 

There are two other characteristics of radio which develop 
from its accessibility. A national network may reach into 
homes all over the country but if it does so it must confine its 
appeal to a general one. This national quality prevents it from 
appealing directly to local interests and experiences. Theoreti- 
cally, of course, the potential audience is great enough for a 
nation-wide program to be beamed at special groups such as 
fishermen, students or stamp collectors and still reach a sizeable 
number of people. But since the aim of the networks is to reach 
as many people as possible at a given time, their specialized 
appeals are confined to large groups such as farmers or house- 
Wives who are known to constitute the majority of listeners at 
certain times of the day. Appeals to smaller groups are left for 
the local stations which cannot hope to compete with the net- 
works on their own ground. The coming of frequency modula- 
Чоп might bring many changes in this respect. 

The discussion of the nature of the problems met in the field 
of research and some of the techniques developed to meet them, 
may be sufficient to bring the reader to two conclusions already 
evident to social scientists concerned with radio. First, there 
15 a growing awareness of the necessity of systematizing the 
knowledge and experience accumulating in this field: a convic- 
tion that such self-conscious rigorization of procedures will be 
of value not only in the field of radio research alone, but to the 
Science of communications in general (and perhaps even to 
other fields of social research). Secondly, as this formulation 
and formalization of procedures and problems proceeds, the 
Sociologist and psychologist working in radio research become 
Increasingly humble about what they do not know. But even 
at this comparatively early stage of their development, radio 
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research activities have already stimulated magazine and news- 
paper publishers to do more research than ever before, and we 
possible that in the not too distant future not only will ыз н 
niques and problems Бе exchanged between these two fields, bu 

funds and research institutions may be merged for the greater 


benefit of both. 
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THE DUTIES OF CIVIL SERVICE EXAMINERS AND 
TEST TECHNICIANS 


Tue following check list is one of a series being prepared 
under the auspices of the Society for Public Administration to 
depict what public personnel workers do. It is a modification 
of a list originally prepared by Dr. John M. Pfiffner for the 
Committee on Education of the Society. It is presented here 
in the hope that readers of EDUCATIONAL AND PsvcHoLoGICAL 
Measurement will offer their constructive criticisms before the 
list is included in the final published report. Comments will 
be welcomed by the chairman of the Committee, Mr. Edgar W. 
Lancaster, Office of the Secretary of War, Room 4E 978, Penta- 
gon, Washington 25, D. C. 

It should be noted that all of the duties listed below are not 
ordinarily performed by any one person. The list is intended 
to cover the work which may be done by all workers engaged 
in merit system examining. A list of the main areas of activity 
15 given first, followed by the more detailed outline. 


Main Topics 


І. Plan examinations? 
II. Construct tests. 
III. Supervise the scoring of tests. 
IV. Evaluate training and experience. 
V. Supervise the administration of tests. 
VI. Supervise the conduct and scoring of competitive oral 
interviews. 
VIL Establish registers of eligibles. 


VIII. Serve as consultant for the service rating program. 
a 


un tem “examination” is used in a broad sense to indicate the entire pro- 
Or series ы ich a person’s qualifications are evaluated with respect to а position 
Or an Positions. An examination may include a test or tests, an oral interview, 
Appraisal of training and experience, either singly or in any combination, 
371 
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IX. 


X. 


Participate in establishing classification specifica- 
tions. 
Conduct research on examinations. 


Detailed Outline 


I. Plan examinations. 


A. 


Assemble data concerning the duties of the positions 
for which examinations are to be conducted. 
1. Consult class specifications. : 
2. Secure additional job descriptions from operating 
officials. 
Search for additional job standards. 
Interview supervisors, foremen, and workers. 
Observe actual work being done and do some of 
the work, if practicable. 
6. Read laws, regulations, directives, manuals and 

books which have a bearing on the job. 
7. Prepare summary showing 

a) Duties. 

b) Knowledge necessary. 

c) Skills used. 

d) Personal characteristics required. 
In the light of information obtained in Step I-A, 
determine the minimum requirements for taking the 
examination, if any, within the limits set by legis- 
lation. 
Outline the nature of the examination considered 
to be most suitable for selecting qualified workers: 
Within the limits set by legislation, and usually after 
consultation with experts in the field, determin? 
whether it shall include an evaluation of training 
and experience, a competitive oral interview, a” 
written or performance tests, giving due considera" 
tion to any pertinent experimental data which maY 
be available. 
Determine weights to be assigned the major part 


de я : : n 
of the examination, taking all pertinent informato 
into account. 


кө 
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E. Prepare the copy for an announcement of the exami- 
nation for the printers, or pass on the requisite infor- 
mation to those in the organization charged with the 
responsibility of issuing announcements. Include 
information on the following topics: 

Title of position. 

Grade and salary of position. 

Location of employment. 

Hours of work. 

Detailed explanation of education and experi- 

ence requirements. 

General information. 

a) Whether written tests, performance tests, 
and competitive oral interviews will be given. 

b) Provisions of regulations regarding qualifica- 
tions statements. 

c) General eligibility requirements in regard to 
citizenship, veterans’ preference, etc. 

d) Information as to how and where application 
should be made. 

e) Weights to be given the various parts of the 
examination. 

II. Construct tests according to plan. 

A. Construct written tests. 

1. Select promising existing test items from file, 
noting statistical history resulting from item 
analysis. 

2. Edit old test items to suit current need. 

3. Write new test items, observing the best psycho- 
logical and psychometric techniques and proce- 
dures. In the case of achievement tests: 

a) Read widely in subject-matter field. 

b) Collect background material. 

c) Confer with other personnel technicians. 

d) Ask subject matter experts to submit test 
material. 

e) Train subject matter experts in test con- 
struction. 


REN 


D 
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III. 


IV. 
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Supervise the administration 


A. 


, B. 


C. 


D. 
E. 


4. Check on the final content of the test. | 
a) Check items for possible defects such bs 
faulty phrasing or the presence of speci 
determiners. oe 
b) See that items are of appropriate sitio 
c) In the case of achievement tests, make ee 
that concepts within the required areas ү 
adequately sampled and that each area і 
appropriately represented. is 
d) Check to see that repetition of concepts 
avoided and that the content of one е ss 
does not help in answering other questions 
€) Check achievement test items with operating 
officials and subject matter experts. di- 
5. Assemble the test in final form with written i 
rections, instructions for administration, answe 
keys, and directions for scoring. EM 
Develop performance tests, when called for, on y 
basis of the study made in step I-A, giving due pes 
sideration to ways of measuring the process of Pe 
formance as well as the final product. 
of tests. ` 
table room or rooms. 
fied proctors. d 
pencils and necessary арр 
ratus ready, with precaution taken to prevent tes 


from being inspected before they are given. 
Train proctors. 


Administer tests. 


Arrange for the use of sui 
Arrange for enough quali 
Arrange to have tests, 


Supervise the scoring of tests, 


A. 


B. 


; к 5 jec- 
Supervise the preparation of scoring keys for obJ 
tive-type questions, mu 
Plan the scoring procedure for questions which 


. . A 3 DORIA an 
not objective, taking steps to insure reliability 
uniformity of Scoring. 


Di estss 
Develop the scoring procedure for performance t 
if used. 


Train scorers. 


VI. 
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Determine weights to be given parts of the test, 
taking all pertinent information into account. 
Obtain total scores in accordance with the plans 
developed for the examination. 

Transmute total scores to a basis appropriate for 
combining with any other measures of qualifications 
which may be used, with due consideration for any 
legal requirements which may exist. 


Evaluate training and experience. 


А. 


Determine the amount of credit, if any, to be given 
various types of experience, obtaining the advice of 
experts in the field and making use of validation 
studies, where possible. 

Determine the amount of credit, if any, to be given 
various types of training on the basis of the advice 
of experts in the field and the conduct of validation 
studies, where possible. 

Develop a schedule to be used in evaluating training 
and experience in accordance with the credit system 
developed. 

When the rating is not done by a rating technician 
or subject matter expert, train clerks in the use of 
the form for evaluating training and experience and 
supervise them in applying the procedure. 

Decide special questions which arise in connection 
with evaluating training and experience. 
Transmute raw scores on training and experience to 
a base appropriate for combination with other mea- 
sures, with due consideration for any legal specifica- 
tions which may exist. 


Plan and supervise the conduct and scoring of competi- 
tive oral interviews when such interviews are employed 
by the merit system. 


A. 


B. 
СЄ: 


Determine characteristics to be measured by the 
oral interview as distinguished from those covered 
in other parts of the examination. 

Develop instructions for interviewers. 

Develop rating scales for factors to be observed in 
the oral examination. 
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D. 


E. 


Outline desirable qualifications for members of inter- 

viewing boards. 

Train members of interviewing boards. 

Develop and apply a method of obtaining scores 

based on competitive oral interviews. 

1. Obtain a score for each interviewer's report. 

2. Combine the scores from all interviewers for each 
applicant, taking necessary steps to insure that 
ratings are given equal weight. 

3. Transmute the scores to a standard scale appro- 
priate for combining with other measures, with 
due consideration for legal requirements. 


Establish registers of eligibles. 


A. 


D. 
Е, 


Serve as consultant for serv 


A. 
B. 


C. 


If weights were not previously established, deter- 
mine the weights for the component parts of the 
examination, taking all pertinent information into 
account. i 

Combine component parts of the examination in 
order to give established weights to the various 
parts. 

Seta passing point and transmute the scores to the 
standard grading system in use. е 
Adjust final scores for special preference groups; if 
any. 

Supervise the Preparation of the register with the 
eligibles placed in the order of final score. 


ice rating program. 

Assist in developing the rating schedules to be used 
and the method of scoring them. | 
Participate in the analysis of the results from service 
ratings. 

Assist in handling 


a of 
problems arising from the изе 
service ratings. 


IX. Participate in establishing classification specifications. 


A. 


Conduct studies on the relation of minimum require 


ments of education and experience to effectiveness 0 
job performance. 


$ TN is 
B. Consult with classification analysts on the analys 


| 
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of required knowledges, skills, and abilities in mean- 
ingful, measurable psychological terms. 

On occasion, apply measurement techniques to the 
evaluation of factors determining the allocation of 
positions to specific grades. 


X. Conduct research on examinations. 
A. Conduct validation studies, preferably before tests 


C. 


D. 


are used, if the need for maintaining the confidential 

nature of the tests allows. 

l. Give to a group or groups of persons comparable 
to those to whom the test will be applied. 

2. Determine criteria against which the tests should 
be validated and develop reliable measures of the 
criteria, when possible. 

3. Obtain measures of the relation between the 
tests and the criteria. Make a study of the 
validity of the test items. 

4. Revise the test or scoring procedure in the light 
of studies of validity. 

5. Continue validation studies through investigat- 
ing the relation of test scores to subsequent per- 
formance on the job, with a view to discovering 
principles to be used in constructing valid tests 
for similar positions in the future. 

Check on the reliability of tests and conduct studies 

leading to the construction of tests of adequate relia- 

bility. 

1. Obtain estimates of reliability of individual tests. 

2. Conduct item analyses to determine internal con- 
sistency of sets of items. 

3. Conduct studies designed to help in eliminating 
items which tend to lower the reliability of tests. 

Make item analyses of tests used and set up a sys- 

tem for maintaining records of the performance of 

items for various groups and examinations. 

Conduct research on miscellaneous measurement 


problems. 


THE ARRANGEMENT OF CHOICES IN MULTIPLE 
CHOICE QUESTIONS AND A SCHEME FOR 
RANDOMIZING CHOICES 


CHARLES I. MOSIER Ax» HELEN С. PRICE 
State Technical Advisory Service, Social Security Board 


In the construction of objective test items in multiple-choice 
and allied forms, the arrangement in order of correct answer 
and choices is often a troublesome chore. Some test writers 
prefer certain positions for the correct choice, finding that items 
having the correct answer in a position other than as the first 
or last choice increases the difficulty of the item;? most, how- 
ever, resort to one device or another to secure a systematic, 
truly random arrangement. When the location of the correct 
choice in the series of choices is left to the whim of the test 
Constructor, personal position-preferences will almost inevita- 
bly result in a preponderance of correct answers falling in one 
Position. Moreover, since distracters tend to be written in 
order of plausibility, with the last distracter often written as a 
desperate final effort, a randomization process should extend 
beyond the correct choice to the incorrect ones as well. The 
Present paper presents a “randomizer” for five-choice items, a 
discussion of its use, and a simple method by which other simi- 
lar aids may be constructed. Some of the situations in which 
1t should not be used are also considered. 

In writing multiple-choice items, we at the State Technical 
Advisory Service follow certain mechanics designed to simplify 
the Writing process and to provide needed controls on quality. 
After the premise or the question has been formulated, the in- 
SENS answer is always written first, with the incorrect choices 
tellowing, This practice of having the intended answer written 


4 McNamar: W. J. and Weit: E. *The Effect of Choice Placement on the 
Difficulty of Monod Choice Questions” Journal of Educational Psychology, 
VE (1945), 103-113. 
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as the first choice has a number of advantages. It insures that 
a correct choice is included and that the item writer, in his zeal 
to prepare plausible distracters, does not end with five plausible 
choices—all wrong. (Such things have happened.) More- 
over, in case of any later doubt that the answer indicated on the 
scoring key may not be the one intended, a quick check against 
the original draft will remove any question about the writer's 
intent. Editorial review and checking are also facilitated. 

After the items are reviewed for authenticity and edited for 
grammatical construction and technical form, the alternative 
answers are assigned their final, random order by use of the 
table attached. The table, constructed for five-choice ques- 
tions, shows the 120 permutations of the numbers one through 
five. In preparing the table the permutations were written 1n 
systematic, cyclic order and each permutation was assigned a 
sequence number from one through 120. Each permutation 
was then assigned, as its final position in the table, the order 1n 
which its sequence number occurred among the last three digits 
of a nine-place table of logarithms. 

The use of this table has these advantages over other SY9" 

tems of randomization: (1) only the numbers actually used 
need be considered; (2) there are no repetitions or omissions 
of choice numbers for any item; and (3) the order of all five 
choices 15 given simultaneously. In applying the table, the item 
writer assigns to each successive item the choice patterns in the 
order in which they occur, beginning each new group of items 
where the previous one left off; every choice pattern is thus us¢ 
once before any pattern is repeated. 
The present table can be used for three- and four-choic* 
items as well. Such use, however, loses the principal advan- 
tages enumerated above and it would seem far simpler to use 
the same procedures in constructing similar *randomizers" for 
those types of items. 

For certain types of items the choices should not be rando? 
ized. When the choices represent selections from a meaning!" › 
ordered series, e.g., dates or magnitudes, it is far less confusing 
to the candidate if they are arranged in their natural order. 
Even where the order of choices is fixed, e.g., by a series ? 
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dates, the randomizing table can be used to advantage in locat- 
ing the correct choice objectively, thus determining the number 
of dates the item writer can select which should precede and 
follow the correct one. By using the table the item writer can 
select the choice pattern, assign to the correct choice the first 
number in the pattern, and distribute the distracters around 
the choice. Thus, if the choice pattern were 2 4 3 5 1 and the 
question were: 

""The year in which the Pilgrims landed at Plymouth Rock 
is”: the intended answer would be choice 2 and the completed 
arrangement of choices might be: 

(1) 1607; (2) 1620; (3) 1628; (4) 1636; (5) 1776. 

A predetermined choice pattern should be used, of course, 
only when the incorrect choices are selected because of their 
association with the question asked; it is more important to 
have effective distracters than to follow a prescribed order. 

Another situation in which the choices should not follow a 
randomized pattern is that in which the choices include any of 
the numbers one to five as answers. In these items, the number 
of the correct choice should be the same as the choice itself; 
thus: 

“The reciprocal of .25 is: (1) one (2) two (3) three (4) 
four (5) five." 

It is sufficiently confusing to the candidate to have to 
answer the question without having to remember, as he might 
if the choices were randomized, “The answer to the question is 
four but ‘four’ is choice number 3 and it is not the answer, four, 
but the number of the answer, 3, which I must mark in the 
answer booklet.” The problem posed by this type of answer 
can, of course, be met by shifting from the designation of choices 
by number to the designation by letter. The use of letters, 
however, has disadvantages and the other solution seems 
preferable. 

There is another situation in which it seems desirable to 
modify the random order of choices. The discussion of this 
Situation is presented here as an hypothesis partially borne out 
by observation rather than as one verified by evidence. When 
the choices presented include a best answer and another which, 
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although not the best, is very nearly as good, the writers have 
observed that the item will very frequently have a negative 
item-test coefficient if the nearly-as-good choice precedes the 
best answer; the coefficient is far more likely to be positive if 
the order of these two choices is reversed. Apparently the high- 
scoring candidates read until they come to the not-quite-so- 
good choice, recognize it as an acceptable answer, give it, and 
turn immediately to the next question. The lower-scoring stu- 
dents, unable to find an answer which their knowledge will рег- 
mit them to identify as correct by recognition, make a careful 
comparison among the alternatives and have a greater proba- 
bility of success than the high-scoring group. Reversing the 
order of the two choices makes the item easier, but tends to 
correct its negative relation with total score. Whether such а 
change will have the desired effect of increasing the discrimina- 
tory power of a particular item must, however, be weighed very 
carefully in the light of the choices in question, the function to 


be served by the item, and the group of candidates for which 
it is intended. 
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THE RATIONALE OF TEMPERAMENT TESTING 
DONCASTER G. HUMM 


Personnel Service, Los Angeles, California 
Temperament 


TEMPERAMENT, according to the dictionary, has to do with 
internal constitution. It also has to do with the peculiar physi- 
cal and mental characteristics that influence an individual’s 
disposition, or the character of mind or mental reactions having 
to do with his behavior. Hence temperament may be consid- 
ered as the pattern or complex of tendencies which determines 
an individual’s behavior. As such, it is made up of traits or 
tendencies to respond in a consistent manner whenever a given 
type of situation arises. Each individual has an abundance of 
traits arising out of the interaction of his original nature and 
his environment; some are chiefly the result of hereditary 
forces; others, of non-hereditary forces; and still others, of 
mixed forces. Some inhibit the effect of others while some tend 
to reinforce others. 

Temperament may be analyzed in many different fashions 
depending upon the psychological attack of the author. Thus, 
Freud analyzes it into the following three tendencies to react: 
(1) toward self-preservation, (2) toward race-preservation— 
or sex, (3) toward gregariousness. Jung, with his emphasis 
Upon attitudes, divides temperament into two great types: the 
Introverts and extroverts. This oversimplification, however, 
should not be accepted as Jung's final analysis, since he sub- 
divides introverts and extroverts into several different sub- 
Classes and also considers the ambivert, the individual who has 
Characteristics that are both extroverted and introverted. 

One of the most useful analyses of temperament is Rosan- 

#5 (1). This analysis has the merit of reflecting the practical 
experience of many prominent students of personality including 
383 


384 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


Leri, Birnbaum, Kraepelin, Spratling, Davenport, Dae 
and Flaubert. Rosanoff’s function has been that of e се gum 
that of integrator plus that of contributor. He has ta dec 
practical experience of these men, added his concept of the i 
trol and directive component of temperament, and E {з 
all of this psychiatric experience into a comprehensive ns 9 
of the field. It should be noted that this analysis is chie Aral 
result of the experience, observations, and research of Dg tens 
trists dating back as far as the 1870’s. We used it as the We 
of the Humm-Wadsworth Temperament Scale because M wt! 
already demonstrated its value to us by explaining prob!e 
of behavior in clinical and industrial use. 


The Characteristics of a Good Temperament Test 


The first characteristic of a good temperament test is d 
itis based upon a comprehensive and valid analysis of tempe K 
ment. As such, a temperament test may be based on -— 
the three analyses we have mentioned or on any analysis whi 


. Н ‚ 3 of an 
is sufficiently comprehensive to cover the reaction pattern 
individual. 


The second characteristic 
adequate standardization. 
with a good sampling of the 
be explored in a comprehensi 


st is 
of a good temperament pe 
Adequate standardization s we 
temperamental traits which MU? 


5 ога!" 
ve analysis. These samples riety 
narily consist of questions or items which may have a va 


of forms. They may be multiple-choice, false-or-true, OF EA 
of the types of items which will bring to light the basic d 
possessed by the test subjects. Each of these test items IP of 
be subjected to a careful item analysis to determine whether 
not it actually elicits the information desired. 

It is important in constructing a temperament test 10 t 
account of the purpose for which the test is to be made Pd 
the attitude with which the individual responds to the test ut 
influence his answers, If the test is to be used for clinical P 6 
poses, the control subjects on whom the test is standard" 
should have the atmosphere of the clinic in which to et n 
to the items or answer the questions. If the test is tO b 
industrial test, the atmosphere which pervades the testing 


ake 
nce 


THE RATIONALE OF TEMPERAMENT TESTING 385 


applicants or workers for industry must also be present when 
the control subjects respond to the questions. However, it has 
been possible in our experience to develop measures by which 
compensations for subjects’ attitudes can be made so as to in- 
crease the usefulness of the test. 

Thus, in the standardization of the Humm-Wadsworth 
Temperament Scale (2) two measures of the subject’s attitude 
were studied: the No Count and the Profile Count. These 
revealed a tendency on the part of the subject to report his 
temperamental tendencies in an atypical manner or to over- 
report them or to underreport them. In the Manual of Direc- 
tions (3) which accompanies the Human-Wadsworth Tempera- 
ment Scale, statistical compensations are reported to make it 
possible to consider overreported and underreported Scales as 
though they had been typically reported. This subject is 
further considered in the Manual of Interpretation (4). There 
has also been provided a Nomograph (5) to make it possible 
to make these compensations more easily. A simple explana- 
tion of the compensations is also included in Personnel Evalu- 
ation Method (6). 

After the items (or questions) of the test have been evalu- 
ated the next step is the construction of norms. In this regard, 
temperament tests are very different from other types of tests 
such as interest inventories, intelligence tests, skill tests, and 
the like. It is very difficult, if not impossible, to construct a 
temperament test with only a single set of norms. This arises 
out of the peculiar nature of temperament. 

In intelligence-test construction, the objective is to find a 
Measure by which the subject may be compared with the whole 
Population, so the procedure is to standardize the projected test 
9n a control group which represents as nearly as possible a 
Cross-section of the population. In temperament-test construc- 
tion many of the tendencies we are trying to measure are very 
difficult, and perhaps impossible, to identify in average well- 
controlled individuals by any means available before the test is 
made. Moreover, some of these tendencies occur with less fre- 
quency than do others; in any survey of a cross-section of the 
Population, they appear as statistically unimportant, but in the 
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individuals in whom they are strong they have the utmost 
importance. 

In standardizing the Humm-Wadsworth Temperament 
Scale we hit upon the device of using control subjects, selected 
by case study, who represented extreme examples of the ten- 
dencies we wished to measure either by possessing the tenden- 
cies to a high degree or by not possessing them at all. The 


following tabulation will illustrate our use of such control 
groups. 


Control Groups Used in Standardizing the Humm- 
Wadsworth Temperament Scale 


Components Plus Groups Minus Groups 
“Normal” Strongly “Normal” Subjects State Hospital Patients 
Hysteroid Habitual Criminals Self-Sacrificing Persons “ss 
Manic | Excitable, Emotional Subjects Subjects lacking Manic Trai 
Depressive Strongly Depressive Subjects Subjects lacking Depressive 

Traits т. 
Autistic Shy, Seclusive Subjects Subjects lacking Autistic 
Traits y 
Paranoid Aggressive, Opinionated Subjects Subjects lacking Paranoid 
7 5 4 1 Traits А А 
Epileptoid ^ Subjects given to Epileptoid Subjects lacking Epileptoid 
Tendencies Traits 


The Scale as developed in this way then gave us a descrip- 
tion of the individual's disposition, a measure of his menta 
health, and a comparison of his tendencies to react with those 
of other typical groups. Thus, a temperament test describes 
the disposition of the subject, 
mastery and self- 


with the reaction 


estimates his powers of seli- 
control, and compares his reaction pattern 
pattern of other subjects of known characte! 
istics. Having provided for such a picture of test subjects by 
comparison of their scores with those of specially selected sub- 
jects we proceeded to learn the meaning of our scores in terms 
of the average, that is the general, population, This was Y 
plished by giving the test to a large group of adults (all of t 

employees, from president to unskilled laborers, of a compa"? 
which had not previously used tests). This group is probab y 


" ye 
not a perfect cross-section of the Whole population but we hav* 


good reason to believe that it is a satisfactory sampling. us 
The distribution of scores afforded by this survey Бат fis 
information as to the average strength in well-adjusted 29" 


"s 


THE RATIONALE OF TEMPERAMENT TESTING 387 


of the tendencies measured. We found, for example, that ten- 
dencies to be sociable, cheerful, active, and emotionally respon- 
Sive to the environment are relatively common, while tendencies 
to be conceited, suspicious of the motives of others and stub- 
bornly fixed in one’s opinions, are fairly rare. 

We are frequently asked by personnel men and by students 
why we cannot provide for an overall score which might be 
taken as a general measure of good or poor temperament. We 
do not do this because an important consideration in the use 
of temperament tests is the identification of patterns of be- 
havior tendency. Thus, such a test really is a battery of tests 
rather than a simple measure. The interrelationships among 
the various measures included in the battery are quite certainly 
More important than the strength of the individual components 
of temperament considered separately. This consideration of 
temperamental patterns or syndromes enormously complicates 
both the construction and interpretation of temperament tests. 
Asa result, the problem of making a temperament test becomes 
ап expensive and time-consuming project, and the problem of 
interpreting the completed test is one which requires the acqui- 
Sition of special skills. I suspect that all types of tests would 
gain in usefulness if we would pay more attention to the spe- 
cialized problems of interpretation each type presents. 

‚ I have mentioned the problem arising out of the attitude 
with which the subject approaches the test situation. In intel- 
18епсе testing and skill testing it may be assumed that most 
Subjects will do the best they can—except, perhaps, for such 
Situations as those in which a criminal might feign feeble- 
mindedness or a soldier try to conceal a skill which would lead 
to an undesired assignment. A 

П temperament testing, however, all sorts of complexities 
affect the subject’s responses. There are, of course, no “right” 
answers or “wrong” answers. Each answer will be true for some 
Subjects and untrue for others. It is often supposed that the 
©xpected or favorable answer would be easily recognized and 
Would be selected by all or most applicants for jobs, but this 

oes not happen. Some subjects seem to be more suggestible 
than others, either positively or negatively; some seem to lean 
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over backwards to claim unfavorable traits; others seem to 
deny the possession of even desirable characteristics. A suc- 
cessful temperament scale must include in its scoring and inter- 
preting procedures means of taking account of these tendencies. 
We have found that certain relationships among the scores 
reveal the effect of these attitudes, and compensation for such 
attitudes can be made. 


Use of Temperament Tests 


As noted previously, a good temperament test may be used 
for a prediction of behavior since it will reveal the status of the 
subject's mental health, it will describe his disposition, and it 
will compare his behavior tendencies with others. However, à 
temperament test cannot be used as a prediction of behavior 
unless the situation in which this behavior is to occur is care- 
fully taken into account and unless the other factors of person- 
ality, aside from temperament, are also taken into account. 
This follows from the fact that a temperament test reports ten- 
dencies—tendencies which are operative only in the presence 0 
trigger situations. Thus, a temperament test should be so con- 
stituted as to report the probable behavior of an individual 
when he is free from undue strains and also report his probable 
behavior when he overcompensates for strains. 

All this makes the estimate of the situation and the estimate 
of other factors influencing behavior, as summarized in the est!- 
mate of probable strains, an important consideration in the use 
of temperament tests, 

The situation in which an individual is placed may or таў 
not be of such a nature as to be conducive to a tranquil, accep- 
table adjustment of the individual. [f it may be taken for 
granted that the individual is in a compatible, sympathetic, an 
kindly atmosphere, it may be taken for granted that strain 1” 
such a situation will be reduced to a minimum. If, however 
the situation has anything in it which is likely to put the 1n E 
vidual on the defensive or likely to give rise to contention ^y 
other forms of unpleasantness, it can be predicted that the iie 
vidual will undergo strain. Thus, it follows that the finding 
of a temperament test alone are not sufficient to predict ho 


UR 
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an individual will respond in any given special situation. For 
example, it is not possible to predict from the findings of a tem- 
perament test how well a student will adjust to college unless 
it is also possible to predict how well the student will like the 
college atmosphere, how suitable his course will be for his apti- 
tudes and interests, and how well he will be received. Similarly, 
it is not possible to predict how well a worker will get along with 
a group of workers unless it is possible to predict how well he 
will like the group, including his boss, how well the group will 
like him and get along with him, and how well the job will fit 
him. А 
There are several factors in the constitution of the indi- 
vidual, aside from his temperament, that have an influence on 
his behavior. Some of these are in the field of aptitude. For 
example, if an individual is placed in a business situation where 
his intelligence is not adequate, one must expect an undue strain 
to result. If he is placed where his intelligence is so superior to 
the job that it is very incompletely utilized, one must expect 
another sort of strain—that of boredom. This reaction is also 
to be expected with reference to skill. A highly skilled worker 
Placed in a job which makes demands for mediocre skill is likely 
to become dissatisfied and get into mischief. A worker who is 
Placed in a position which requires more skill than he possesses 
is likely to become discouraged or defensive or in some other 
Way to compensate for his feelings of inadequacy. S 
Health is also an important consideration in estimating 

Strain, For example, a man of super-abundant energy with 
Considerable pressure of activity cannot be tied down to an 
Mactive job without the expectation of some over-compensation 
Оп his part. Likewise, an individual who is struggling in a job 

Syond his strength is likely to suffer, not only with respect to 
his physical health, but also with respect to his mental health. 

_ It follows that the prediction of behavior can be accom- 
Plished by the use of a temperament test if the findings of that 
test are supplemented by findings of other sorts—probably in- 
cluding non-test data as well as test data—to predict the 
amount of strain the individual may be expected to endure in 
the situation or situations under consideration. 
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Temperament Testing in Clinical Practice 


Valid temperament tests are useful in studies of individuals 
for vocational guidance, educational guidance, and problems of 
social and marital adjustment. In many instances a good tem- 
perament test will indicate whether or not a personal problem 
will be complicated by a poor state of mental health or by an 
insufficiency of self-control or self-mastery to direct dynamic 
temperamental qualities. It should be the practice of a psy- 
chologist, however, to refer problems of mental health to psy- 
chiatrists for examination. Whenever there is any question 0 
psychosis, psychoneurosis, or a psychopathic state, it is neces- 
sary to consider not only the behavior of the individual but also 
the nature of the handicap or disablement. This makes it 
essential to secure a medical diagnosis as well as a psychological 
diagnosis. Psychiatrists only are equipped professionally to 
consider both these phases. 

. A painstakingly thorough study of personality is required 
in the consideration of the readjustment of the individual. +t 
seems reasonable that the minimum points to be covered are 
the following: (1) family history, at least as far back as the 
grandparents, in which noteworthy achievements and handi- 
caps are taken into account; (2) personal history from concep- 
tion, including childhood, adolescence and adulthood; (3) 4 
particularized history of difficulties in making adjustment 
especially the failures in school and social and job adjustments; 
(4) a physical examination by a competent physician; (5) 2 
preliminary mental examination by means of a valid tempera- 
ment test followed by a verification of the results in a person? 
interview or by psychiatric examination; (6) an interest exam" 
nation by a standardized interest inventory; (7) examinatio? 
of skills and aptitudes by competent tests; (8) examinations 
intelligence tests—preferably by an individual intelligence test; 
(9) the analysis of all of the data obtained in steps one to eig Я 
and а report to the subject; (10) a written summary and report 
to the subject. j 

The use of such a procedure is very likely to be effective ш 
substantiating and explaining the individual tests by the гези ® 
of the tests in other fields. Such a procedure is more t г 
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likely to bring to light the extent to which the individual has 
undergone strain, the character of that strain, and its probable 
effect upon his temperamental integration. 


Temperament Tests in Industry 


Temperament tests in industry are valuable to supplement 
aptitude tests and data obtained by non-testing procedures. 
Aptitude tests tend to reveal to the technician what the indi- 
vidual can do; temperament tests, what he will do; and interest 
tests, what he likes to do. The integration of testing methods 
and non-testing methods in.a routinized procedure is very likely 
to prove more effective than the use of either test procedures 
Ог non-test procedures alone. A good industrial appraisal pro- 
gram probably should include the following: 

(1) A standardized application form or job-specification- 
and-qualification sheet. This form should contain spaces for 
background, training, experience, job titles, and job duties. 

(2) Intelligence tests; if group tests are used, at least two 
should be included. When possible, one of these should be a 
timed test and опе an untimed test. Some individuals do not 
respond well to timed tests. 

‚ (3) A temperament test; this test determines the indi- 
vidual's self-mastery and self-control, the strength of his tem- 
Peramental characteristics, and his behavior tendency pattern. 

(4) An interest inventory; this measure determines whether 
Ог not the individual's interests are such as to make him con- 
tented in the type of work being considered. 

(5) Skill or aptitude tests; skill tests are to be preferred 
Where the individual is already trained for the contemplated 
Job. Aptitude tests are to be preferred where the individual 
1S a trainee, 

(6) Physical examination by the company physician. 

KS) Summary of all of the data considered in the foregoing 
SIX procedures, a listing of assets and liabilities with regard to 
the Job, and a statement of job risk. 

Such a set of procedures as this can be so routinized as to take 
ess than three hours’ time. The fact that many of the tests are 
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group tests makes it possible to test many individuals simul- 
taneously. However, the most important feature of such a set 
of procedures is its thoroughness. After all of these points have 
been covered, it is possible to have such an understanding of the 
potentialities of the worker as can be used for selection, place- 
ment, counseling, supervision, and readjustment. 


Summary 


Temperament is one aspect of personality, but not the whole 
of personality. It is the pattern or complex of tendencies which 
express themselves in behavior in the presence of trigger situ- 
ations. 

The measurement of temperament requires: (1) a valid and 
comprehensive analysis of temperament as a base of departures 
(2) items or questions which adequately sample the field © 
temperament; (3) adequate item analysis; (4) norms whic 
afford a description of temperament and comparisons with the 
population; (5) provision for dealing with atypical response 
attitudes. 

Temperament tests may be used for the prediction of be- 
havior when other pertinent facts are known; that is, when che 
environmental strain can be estimated. They cannot be use 
for such prediction unless environmental strains are considered. 
(Incidentally, environmental strain cannot be taken as а СОЛ” 
stant. Even the conditions of combat represent for some men 
a challenge or opportunity or release, while for others they 
represent only danger or sorrow or frustration.) : 

Temperament tests, Properly used, can be valuable aids t° 
the clinical and industrial psychologist for the information they 
give with respect to mental health, temperamental integratio™ 


: "ues e- 
strength of various temperamental characteristics, and 
havior patterns. 
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MECHANICAL ABILITY, ITS NATURE AND 
MEASUREMENT. П. MANUAL 
DEXTERITY 


J. R. WITTENBORN 
Yale University 


Introduction 


The factor analyses of test samples employed in studies of 
mechanical and motor abilities by Harrell (3) and Wittenborn 
(6) have shown that the variables may be classified on the 
basis of their interrelationships. These classifications, or fac- 
tors, offer a functional basis for the definition of abilities. In 
the analyses of mechanical ability these factors appear to be of 
two types and for the sake of simple designation may be given 
the superficially descriptive labels of “mental” and “motor.” 

ese rubrics are, of course, not explanatory, but they are 
appropriate insofar as the variables contributing to the “men- 
tal" abilities (scholastic, spatial visualizing, and perceptual) 
are considered to be independent of the exact mode of expres- 
Son, . Tests contributing to the “motor” abilities (dexterity, 
repetitive movement, and steadiness) appear to be peculiarly 

pendent upon the quality of muscular performance. f 
‚ The present paper is concerned chiefly with *motor" abili- 
ties, particularly those which may be called “manual.” It is 
ased primarily on data from the Experiment Proper of the 
Innesota Mechanical Ability program of research (4). The 
Ihnésota program had two aims: one was to predict “me- 
chanical ability” for a group of junior high-school boys in shop 
Courses; the other was to understand the general nature of 
mechanical ability, something about its origins, and the condi- 
tons for its development. As a part of the Minnesota study 
ОЁ the nature of mechanical ability, the following variables from 
the Experiment Proper were intercorrelated: 
395 
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1. Age 26. Father’s mechanical operations 

2. Otis, ІО. 27. Tools owned by son 

3. Packing Blocks 28. Tools owned by father 

4. Card Sorting 29. Things done questionnaire 

5. Minn. Spatial Relations 30. Mechanical occupations prefer- 

6. Paper Form Board ences 

7. Stenquist Picture I 31. Academic preferences 

8. Stenquist Picture II 32. Interest Analysis Blank (old) 

9. Minn. Assembly 33. Gymnasium ranks 

10. 100-yard dash 34. Academic grades 

1l. Back dynamometer 35. Garfield's Agility battery 

12. Right-hand dynamometer 36. Minn. Agility battery 

13. Steadiness 37. Interest Analysis Blank (new), 

14. Left-hand dynamometer 38. Shop operations quantity-quality 

15. 25-yard hop criterion 

16. Spirometer 39. Education of father 

17. Broad jump 40. Education of mother . 

18. Height 41. Mechanical ability rating of 

19. Weight father's occupations 

20. Shop operations quality criterion 42. Mechanical ability rating of other 

21. Shop operations information cri- ancestors’ occupations , 
terion 43. Barr scale ratings of father's 

22. Cultural status occupations 

23. Literary interests 44. Barr scale ratings of other ances- 

24. Recreational interests tors' occupations 

25. Son's mechanical operations 45. Otis, mental age 


An examination of the intercorrelations revealed that 
numerous variables, such as numbers 22, 23, 24, 27, 28, 29, and 
30, bore no important relationships with other variables. Cer- 
tain other variables, such as 35 and 36, 39 and 40, 41 and 42, 43 
and 44, tended to form independent couplets and as a conen- 
quence were of no general interest. Mental age and other vari- 
ables relating to academic status were of no interest in the 
present study, and the sole steadiness test, variable 13, showed 
no important relationship with any of the other variables. 
Certain of the variables, however, showed significant interrela- 
tionships, and their nature suggested that further scrutiny 
might afford additional insight into the nature and organization 
of mechanical ability. These Promising variables and thet 
intercorrelations are presented in Table 1, 


An Analysis of the Minnesota Data . 
The 16 variables for which intercorrelations are shown 1? 
Table 1 were selected with certain expectations. It was Þe- 
lieved, for example, that the pattern of their intercorrelation? 
might confirm the tendency for measures involving a high de- 
gree of manual dexterity to form an independent function? 
classification (6). It was expected, moreover, that an analys!$ 
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TABLE 1 
Intercorrelations of 16 Selected Variables (N = 100) 


А 
m I 32 
у ы м Б = 
Д чевь B eB 3.2 
оз 2222 он "БЕ" 
= 5 © ER RE s SES .25, SE EF 
ПШ Е Za 5 o e g ASETE SO g <8 
ы S$ 05 = 3 8 < жеар ое ы oe Pas 
£ a we Td "ES Ej > АЧЕШБЕ g Бор = BT 
ч т Es & S d E y ERSE B Pss БЕ 
= 3 RI v = А. ч - LU popes 
& O EZ $ à à E SHASAG ш EFEQS 5m 
3 4 5 6 7 8 9 H 12 M 16 18 19 20 25 37 
3 
4 52 
5 34 23 
6 .14 14 63 
7 18 .10 42 .37 
8 21 24 .39 .30 .54 
9 .30 .13 .56 49 46 40 
11 09 .$ 1-01 25 dl as 


12 -.03 -.04 —.05 –.10 .08 -.12 .04 .66 

14 -.06 ~ 106 –.09 —.09 .15 —.10 .06 .70 .84 

16-07 03-01 .03 .16 -.02 04 54 .50 .60 

18 ~.02 -.09 01-08 .16 -.11-.01 48 .58 .60 .72 

19 -.09 -.11 - (01 -.05 18-04 04 59 67 68 .74 .78 

20 .26: 19 53 52 24 31 155 04 05 02 .03 -.09 .02 

25 00-12 22 24 24 19 40.1 15 09 .04 .09 10 .30 

37 12 09 46 3932 28 42 08 09 .04 03-01 03 .64 .30 


of the selected intercorrelations would contribute to our under- 
Standing of the general nature of the dexterity factor. It 
Seemed to be particularly desirable to know to what degree 
measures of strength and physical development contribute to 
an ability such as manual dexterity. 

The intercorrelations were subjected to a centroid analysis 
and four factors were extracted. No residual significantly 
&reater than zero remained. When the centroid matrix, Table 
2, is postmultiplied by the transformation matrix, Table 3, the 
orthogonal rotated factor matrix, Table 4, is produced. 

Although an orthogonal solution is given to the present 
Problem, it is apparent that Factors I and II are not truly inde- 
Pendent. The variables which cluster together to form Factor 
IT have higher loadings on Factor I than on Factor II. It is 
apparent, therefore, that presentation of Factor II as a factor 
Independent of Factor I is not in strict conformance with the 


398 


EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


TABLE 2 
Centroid Matrix 


I II ш IV 
eS с ШЫ ш Т 
1 58 -.72 = 39 
II .39 -J8 E -.83 
ш 67 ‘59 eh ‘00 
IV 25 32 "32 39 
zm. c В „ш 


I Ш ш y 
"а Packing Вой. ы ы сс —— — 
3. Packing Blocks ................. ДӨ: ы 65 б 
4. Card Sorting ....... їзбє vectis 07 жү 1 67 A 
5. Minn. Spatial Relations ......... 02, 0f ‘74 26 51 
6. Paper Form Board ............. -.02 07 70 10 po 
7. Stenquist Picture I ............. .06 39 47 23 42 
8. Stenquist Picture II ............ 217 28 42 37 %8 
9. Minn. А: 25 4 15 63 
11. Back dynamometer ч 48 ‘00 18 73 
12. Right-hand dynamometer | 49 —05 — «07 “30 
14. Left-hand dynamometer ї БТ 1-40 d 71 
16. Spirometer i 07 05  -.10 70 
18. Height : 05)" 403. 16 М2 
19. Weight „зы шаш айя mae ob ari an .84 20 05 —.19 “60 
20. Shop operations quality criterion .. .00 .03 45 19 29 
25. Son’s mech. operations .......... .03 32 4l -.15 47 
37. Interest Analysis Blank (new) ... —02 17 66 01 Ё 


ТАВГЕ 3 


Transformation Matrix 


TABLE 4 


I II Ш 

„Расло: Blacks: „ааваасаа 31 43 39 
Card Sorting ......... 24 26 45 
Minn. Spatial Relations 56 52  -.2 
Paper Form Board .... 46 48  -25 

. Stenquist Picture I . 56 .23 12 
‚ Stenquist Picture II 40 43 23 
. Minn. Assembly .... 59 45 -.10 
. Back dynamometer ... 61 47 27 
. Right-hand dynamometer . 53 -64 .10 
. Left-hand dynamometer ,. = 54 = 68 RIS 
. Spirometer .......... 5 Sj 589 16 
eight 49 -64 -.17 
Weight ......... 5% = —Д7 
Shop operations quality criter 55 47 -.25 
on's mech. operations .......... 35 14  -23 
Interest Analysis Blank (new) ... 50 37 -22 
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numerical solution in the present study. As the factors are dis- 
cussed variable by variable, the data will be presented in the 
form of factorial equations. 

Factor I appears to be a size or a maturational factor. It is 
determined primarily by variables 16, 18, and 19. 


Factor I—Size 

T п ш IV у 
П. Back dynamometer ........ 37 23 -00 -03 37 
12. Right-hand dynamometer ..  .48 24 .00 .00 28 
14. Left-hand dynamometer ...  .52 27 (2.01 .00 320 
16. Spirometer экее еган .69 .00 00 (-).01 30 
I3. Height: «cssc sirena uu 67 00 00 (-) .03 30 
19.. Weight лаин conve 4i 04 .00 (-) .04 21 


Approximately 70 per cent of the total variance of each of these 
variables is found in Factor I and no significant amount of vari- 
ance is contributed by these variables to any other factor. 
Tests 11, 12, and 14, which suggest a strength factor, Factor II, 
actually have most of their common factor variance and ap- 
Proximately 50 per cent of their total variance in Factor I. 
This finding is of interest because in an analysis of data for 328 
youths who were older than the present group it has been found 
that strength and size are independent of each other (2). Be- 
Cause of this, Factor I and Factor II are treated in the present 
Study as independent of each other. The writer offers as addi- 
tional justification for this treatment the consideration that no 
additional understanding of the organization of the variables 
Would result from rigorously defining Factor II as highly corre- 
lated with Factor L? Since the data of the present study do 
Dot call for a strength factor independent of the size factor, this 
Independence can only be considered as hypothetical. It is 
reasonable to find size and strength highly correlated among 
Young boys and to expect these variables to become increasingly 
Independent as maturation is attained. It is hoped that the 
results of this study will have implications for the use of certain 
PESO 

1 Factorial equations are more revealing than simple factor loadings because they 


ie only show how much of the total variance of the test is due to each factor, but 
they also show how much variance is not due to common factors, i.e., how much (и?) 


75 unique to the test in the present sample. The values in the factorial equations are 


equal to the respective factor loadings squared. UN К 

? Actually the present data could be accepted as yielding 3 factors: size-strength, 
SPatial ability, and dexterity. The four-factor solution is somewhat “forced” and 
Justified by the above-mentioned considerations. 
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types of tests in the selection and guidance of young adults. 
It is hypothesized, therefore, that for such young adult groups 
body size and strength of the upper parts of the body are rela- 
tively independent of each other. | 

Factor II, the postulated strength factor, is of considerable 
interest in the present study because its variables do not con- 
tribute to the manual dexterity factor, Factor IV. 


Factor II—Strength, 
I I ш IV p 
11. Back dynamometer ........ 37 23 .00 .03 a5 
12. Right-hand dynamometer .. 48 24 .00 .00 20 
14. Left-hand dynamometer .... 52 27 (2.01 00 á 


The contribution which Factor II makes to certain other vari- 
ables such as the Son's Mechanical Operations variable and the 
Stenquist Assembly tests is meaningful insofar as strength © 
hands among boys would be expected to be associated with the 
use of the hands either as indicated directly by the Son's opera" 
tions questionnaire or indirectly by the Stenquist Assembly 
tests which sample mechanical knowledge. The fact that va?" 
ables 3 and 4, the manual-dexterity variables, do not contribute 
to this factor in any way is taken as additional evidence that 
manual dexterity is a classification of ability quite independent 
of other types of manual ability (6). 

Factor III, the spatial relations factor is defined by the 
Minnesota Mechanical Assembly Test, the Minnesota Paper 
Form Board Test and the Minnesota Spatial Relations Test. 


Factor III —Spatia] Visualization 


I п n у U 
. Minn. Spatial Relations 


. Paper Form Board ... 
- Stenquist Picture І... 
. Stenquist Picture II 


5 

6 

7 

8 я 

9. wats А Ё 
a, Shop operations quality criterion .00 .00 
37. 


. Minn. Assembly 00 06 37 02 40 
| 56 (-).04 71 

. Son’s mechanical operations .... .00 10 17 (2.02 

- Interest Analysis Blank (new) .. — 0 оу — 4 (^ Qo 553 


It is most interesting to observe that variable 20, the ShoP 
Operations Criterion, has all of its common factor variance ар 

over 50 per cent of its total variance in this particular facto" 
In addition, mechanical interests, 37, and Son’s Operations и 


У ec 
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mechanical activities, 25, are also highly correlated with this 
factor. 

The importance of measures of spatial ability as indices of 
mechanical promise is strikingly indicated by the nature of this 
factor. Not only the criterion but interest in mechanical activi- 
ties appears to be quite independent of the three additional 
factors which appear in this study and which might on an a 
Priori basis be expected to contribute to a shop operations cri- 
terion of mechanical ability. 

Factor IV is perhaps the most interesting factor in the pres- 
ent study. 


Factor IV—Manual Dexterity 


I II ш IV U 
3. Packing Block 00 (-) .01 05 42 52 
(afte 5 os д 5 à 


It is defined by two tests which appear to call for a type of 
manual dexterity. However, these tests do not contribute to 
the spatial-visualizing factor which in the light of this study 
18 the mechanical-ability factor. Perhaps more surprising is the 
fact that neither the strength nor the size factors contribute in 
any way to facility in manual dexterity as identified by this 
actor. Although manual dexterity is an ability which has long 
een considered as a definable attribute, prior to the investiga- 
tions of this series its existence as a functional classification of 
ability had not been satisfactorily demonstrated. As a matter 
of fact the data presented in the present and in the preceding 
Study may not be regarded as adequate to define satisfactorily 
an ability such as manual dexterity. This reservation is reason- 
able since block packing and card sorting were principal varia- 
€s in defining this factor in both of these studies. Although 
the studies were done on two samples and the factor occurs in 
two different test batteries, its existence requires further 
€monstration. 


Further Evidence for Identifying M anual Dexterity 
In order to shed more light on the existence and the nature 


of this factor additional data were sought for further scrutiny. 


ata suitable for this purpose were found in the Measurement 
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of Manual Dexterities by Earle (1). He had published E 
intercorrelations of ten measures, nine of which were considere 
to be measures of manual dexterity. The tenth was a criterion 
of mechanical ability, the exact nature of which was not defined 
in his publication. The intercorrelations of these variables for 
a sample of 79 continuation Day-School boys are presented 5 
Table 5° and the nature of each of the performances is indicate 
below: 
TABLE 5 
Intercorrelations for 79 Day-School Boys* 


I п ш VL VL уп уш IX ХШ Crt 


ПИРА» 1.138. 33 32 но 
XII 04 .13 0 40 29 19 % 19 
Coty СО ә" 1177 722 "1, Тї 24 .d6 0 


* Median age, 14 years and 5 months. 


Test I. Tapping movement of forefinger using wrist. A io 
is tapped by the forefinger of the preferred hand Шш 
the non-preferred hand holds the apparatus. А 
Tapping movements of several fingers in ques 
using wrist. The individual taps with each of t fe 
four fingers successively beginning with the be 
finger each time and tapping in order from the litt 
finger to the index finger as quickly as possible. A. 
Test III. Twisting movements of finger and thumb with wWri 
action. In this test the individual is required to tum 
the barrel of the turn buckle until the eye is aS far m 
the barrel as possible and then to reverse the directio" 


of rotation and turn the barrel as quickly as poss! 
until the eye is released. 


Test VI, part I: The individual is required to place 100 pees 


into a 100-hole peg board, picking up опе pee 
at a time. 


Test II. 


r3 B . 4 the 
3 Precise scoring methods used in securing these data are not specified by 
author. 
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part II: The individual is to fill three rows of the 
peg board, halting temporarily between rows. 
Test VII. The individual is requested to place the pegs in the 
peg board using the thumb and each finger (except 
the forefinger) in succession to pick up the pegs. 
Test VIII. Placing pegs in holes under manipulative diffi- 
culties. 

Part I: The individual takes all the pegs from one 
row of the peg board and then replaces them 
keeping them in his hand as he works; 

Part II: The individual extracts all the pegs from 
| two rows of the peg board and then returns 
them keeping them in his hand during the 
process. 

Parts III and IV are conducted under such 

manipulative difficulties as maintaining the 

Ж operating hand full of pegs while working. 

Test IX. Placing pegs in holes which are not visible. In this 
test a tactual exploration is made by the free hand, 
usually the left, while the right hand is used to pick 
up the pegs. The subject is not blindfolded but a 
screen is placed between him and the board. It is 
required that the subject feel for the hole each time 
with his left hand. 

Test XIII. Discrimination between fine and coarse textures 
by sense of touch. A screen is placed between the 
individual and a tactual board upon which strips of 
sand paper are placed and manipulated in such a 
fashion as to permit the individual to attempt to tac- 
tually recognize the match for several different grades 
of sand paper. 


Tests IX and XIII are of particular interest because they 
are relevant to a question raised in the first paper in this series; 
It was found that the digit-symbol-substitution test, which 
would appear to call for no high degree of manipulative dex- 
terity, was significantly correlated with the dexterity factor. 
The identity and the nature of the dexterity factor was put in 


404 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


doubt by this finding. It appeared plausible that the dexterity 
factor is the old hand-eye coordination ability so commonly 
spoken of in earlier studies of mechanical testing. It was Sug- 
gested that a hypothesis that manual dexterity calls for visual 
recognition and discrimination as well as a manipulative ability 
could be tested by screening the manipulative work from the 
testee's field of vision. Test IX suggests the possibility of а 
preliminary test of this hypothesis. ; 
In order to examine this possibility and to seek further evi- 
dence of manual dexterity as a valid functional classification 
of ability, Earle's intercorrelations of the ten variables were но 
mitted to a factor analysis. It was found that two centrol 
factors, Table 6, permitted a satisfactory reconstruction of the 
intercorrelation table. The two factors were then subjecte 
to a single orthogonal rotation and the nature of the rotate 
factors, as shown in Table 6, appears to be meaningful. 


TABLE 6 
Å= 
Centroid Factors Rotated Factors 
——______Centroid Factors ———— Rotated Factors _ 
I m n В 
E исседон на А ы ——— ' Ree 
І Tapping ........... 47 —.44 65 06 
IL Tapping 


VIL. Pegs 
VI. Pegs 


УШ. Pegs (handic 


Р) энем d 
IX. Pegs (not visible) . " 
ХШ. Таса 219) 4 


Criterion 


Factor A calls for the same class of operations as the ballis” 
or repetitive movement factor described in the first paper 
this series. Earle's test IIT which calls for a simple, high’ 
speeded twisting or twirling movement of the fingers 15 high Y 
correlated with Factor A. This finding suggests that the simP и 
repetitive movement of tapping involves the same ability 2 
the more industrially significant repetitive twisting or twirling 
manual operations. Tests for the repetitive movement eer 
may conceivably be of considerable value in selecting worker 


ү 6 Aug nt. 
for certain types of common industrial piece work employ me 


t 
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Factor B seems to call for the same type of manual facility 
or manipulative ability which characterizes the dexterity factor 
revealed by the analyses of the Minnesota data. The tests 
differ from any of those used in the Minnesota study. One of 
them, test IX, does not permit the use of vision. Performance 
on test XIII depends upon accuracy of tactual discrimination. 
The analysis of Earle's data not only confirms the tendency for 
measures of manipulative ability to be positively intercorre- 
lated but contributes to our understanding of this tendency. 
Tests calling for controlled placing, or adjusting movements, 
of the hands are interrelated as if they depended upon a single 
ability or capacity. This ability appears to be independent of 
the visual modality; it may be chiefly dependent upon the 
tactual and kinesthetic modalities. 

The tendency for tests involving manual dexterity to be 
more highly correlated among themselves than with other tests 
is manifested in yet another context. Teagarden (5) has inter- 
correlated two Kent-Shakow scores, Minnesota Spatial Rela- 
tion, two Minnesota Rate of Manipulation scores, and two 
scores from the Cincinnati Pliers Test. The tendency for the 
spatial visualization tests to form a correlation cluster different 


from the cluster formed by the dexterity tests 1s unmistakable. 


Conclusions 


Guidance experts and personnel technicians use tests which 
on the basis of the current statistical classifications are con- 
sidered to be measures of steadiness, repetitive movement, dex- 
terity, and spatial visualization. Yet, with the exception of 
the use of the spatial visualization tests, their procedures are 
justified chiefly on the basis of intuition and not on the basis 
of high correlations with external validating criteria, 1.€., Spe- 
cific industrial performance. The literature abounds with evi- 
dence of the validity of spatial visualizing ability tests for pre- 
diction of mechanical work. Acceptable external validities of 
manual dexterity tests are more rare, however. In general, the 
Most promising validity coefficients for manual dexterity tests 
have been obtained with ratings of supervisors as a criterion. 
Evidence of validity for the practical use of measures of repeti- 
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tive movement and steadiness are practically non-existent in 
literature. . 3 
c suggestion is offered that the common failure to es 
date tests of factors other than spatial visualization and sc d 
lastic ability (which at the adult level may be fractionated de 
other abilities) is probably due to the nature of the criteria t A 
have been employed. Most of the criteria that have been aie 
ployed in the prediction of mechanical ability have been 
samples prepared under unusual competition and other atyp of 
conditions which appear to call for a much higher e 
spatial visualizing judgment than manipulative ability, e.g. тог 
criteria used in the Minnesota study. The so-called all 
aspects of mechanical ability cannot be assumed to be of Ша 
significance simply because their significance has not been ^s 
ously demonstrated by suitable studies. If investigators oye 
ployed such criteria as satisfaction in work, duration of emp tac 
ment in routine operations, speed of work, quality of Pte 
operations, piece work output, breakage, fatigability and ot 
factors of great practical significance in industrial operatio E 
it might well be demonstrated that the motor abilities, part! 
larly manipulative ability could, on the basis of demonstrate 
predicted value, be granted a significant róle in guidance ап 
selection procedures. de 
The term “mechanical ability” does not lend itself to а ae 
quate definition, however. In modern industrial employme A 
there are innumerable different operations which involve 
use of machines, tools and other mechanical contrivances- к. 
appears likely that the successful prediction of satisfactory р » 
formance and good morale in these industrial activities 18 jo 
dependent upon the development of adequate criteria did 
upon the invention of new ability tests. Tt is suggested ү а 
the greatest immediate Progress in the field of meclig 
ability testing depends upon extensive factor analytical stu e 
of interrelationships of criteria of different phases of а: 
operations and at different levels and types of work. Un ost 
tunately such varied criteria would not be available for m à 
groups of industrial workers; certain paid apprenticeship og 
training groups would probably be the most desirable subJ 
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for this research. It is only on the basis of such intensive re- 
search that mechanical ability may be satisfactorily defined. 
Definition of mechanical and manual work on any other basis 
is arbitrary and therefore not likely to be generally applicable. 

The studies of the present series demonstrate the commonly 
observed tendency for intercorrelated psychological variables 
to form clusters, i.e., to permit a somewhat rigorous mathe- 
matical classification of the variables. The question which con- 
tinually arises in a discussion of such studies as these is what 
significance may be ascribed to the classifications. The classi- 
fications which have been established by factor analysis could 
certainly be due to the sampling of the measures which are sub- 
jected to analysis. The sampling could be either a deliberately 
or an unconsciously obtained result. 

The test samples could be a function of our culture. Per- 
haps the human organism is physically capable of an indefinite 
Variety of response patterns. If this were so, the culture could 
be regarded as determining which response patterns are of 
Practical significance and through this mode of influence the 
culture could also determine the pattern of performances sam- 
pled by current psychological tests. In addition to this selec- 
tive effect, it is conceivable that a given culture may actually 
determine the development of abilities. Just as response pat- 
terns are elicited by life experiences, ability patterns may 
appear in a group of individuals in response to the exigencies 
ОЁ existence in the society. These cultural requirements may 
Possess a structure which could be reflected in the organization 
of ability, The appearance of a pattern of ability among all of 
the individuals participating in a culture is expressed by the 
Consistency in the society of intra-individual differences with 
Tespect to the various classes of ability. Currently the most 
Satisfactory explanation of the development of intra-individual 
differences would rest upon learning theory. Learning theory is 
sufficiently well developed to enable us to envisage in a general 
Way the manner in which differences in the environments of 
individuals could favor the development of intra-individual 
differences along lines which would reflect aspects of the culture. 

The great diversity of culture has been observed from group 
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to group at different periods with respect to many important 
attributes. It has not been shown to the writer’s knowledge, 
however, that the factorial pattern of ability varies meaning- 
fully from culture to culture. Until this diversity has been 
demonstrated, the arresting possibility remains that the HE 
zation of ability may be a more or less standard pattern an 
a necessary consequence of the functional limitations of the 
human organism. A careful investigation of cultural differ- 
ences in ability patterns appears to be necessary for the de- 
velopment of a science of human ability. ‘ 
As previously mentioned, validity of tests for the peal 
factors is directly dependent upon the type of criteria employa $ 
Regardless of the nature of the criteria, however, it is frequent'y 
found that specially devised tests of rather anomalous factoria 
composition show higher validities than tests for known — 
The superior validity of tests which are specific to a task inp 
the existence of specific factors practically significant for E 
tasks. The ultimate significance of such hypothetical spec! - 
factors is unknown and probably rests in part upon the nat" à 
of their origin. Since the identifiable, stable factors аге 3P 
parent among individuals at different age levels, it may һе 
inferred that such abilities are relatively stable within t? 
individual and therefore may be validly employed in me 
range predictions. It is possible, however, that specific аы 
ties measured by certain tests are readily acquired in cem 
to specific experiences and may have no great ultimate pT° ; 
tive value despite their value in predicting criteria established 
shortly after the application of the original test. More d 
establishing the long term predictive value of all of our pen 
are greatly needed. Long term predictions are always чий 
reliable than immediate ones. This effect is doubtlessly С" 


а LL H H en d 
in part to the loss or acquisition of readily acquired, trans! 
specific abilities. 


Summary d 

A š an 
On the basis of factorial analyses of the Minnesota da'a iey 
the examination of data of other studies of mechanical 2?!" 7 


ME SRT епі“ 
It Is apparent that а complete assay of an individual's Ро 
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apis for all types of mechanical or manual work would call 
or measurement of at least the following attributes: 


l. scholastic ability 5. repetitive movement 
2. spatial visualization 6. steadiness 

3. perceptual speed 7. strength 

4. manual dexterity 8. size 


The exact organization of the factors at different age levels 
requires further analytical study. The degree to which the 
importance of each of these abilities varies from job to job is 
unknown, but it is subject to critical determination. 
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. _Wueruer by force of tradition, or for reasons of expedience, 
it has been the practice to administer and score group tests of 
ability, aptitude, and achievement in such a way as to yield 
only a time-limit score, defined as the number of items cor- 
rectly answered within a specified length of time. Thus, the 
time-limit score often becomes the sole measure of the behavior 
represented in a test. When a test is “validated” with respect 
to some external criterion, a time-limit score, rather than some 
other type of score, is most likely to be used as the measure 
Which is correlated with the criterion. Likewise, in making a 
factor analysis of a battery of tests, one is most likely to use 
time-limit scores, It is the writers’ belief that the indiscrimi- 
Nate use of time-limit scores is one of the more unfortunate 
tharacteristics of current psychological testing since the time- 
Iit score of a test frequently represents two relatively inde- 
Pendent aspects of behavior: (a) the amount the subject knows 
9r can perform (or in certain cases, the level of difficulty which 
e can reach), and (b) the rate at which the subject works. 
Omewhat at variance with current usage, we shall identify 
these aspects of test behavior, respectively, by the terms level 
and speed. By ignoring the possibility that these two aspects 


A 


of test А : а E 
erf rent rôles in any given situ- 
—_*St performance may play diffe yg 


іт; 
сапа q Paper is a revision of a thesis presented by the first-named author as a 
aate for the M.A. degree at Indiana University, 1943. 
indepen ace terms were employed by Baxter (D, who was able to show a marked 
study, į ence of speed and level in a single omnibus test of intelligence. The present 
» іп effect, extends Baxter's approach to tests of varying content. 
411 
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ation, the applied psychologist runs the risk of obtaining valid- 
ity coefficients lower than those which might be obtained if the 
level and speed components were correctly weighted in the pre- 
diction. For example, if the level score on a test has greater 
validity than the speed score in predicting a criterion, use of the 
time-limit score may tend to mask the potential validity of the 
test by introducing the “dead wood” variance of the speed com- 
ponent. In factor analysis there exists a real danger that the 
primary factors which are found in a set of time-limit scores 
may themselves be factorially complex, that is, that they тау 
consist of both speed and level components. When these рї” 
mary factors are correlated, as is frequently the case, one shou 
not consider the hypothesis that the correlations indicate the 
presence of a general factor of intelligence until it is shown that 
they are not due to the presence of an underlying speed factor. 
It is true that a logical distinction between speed and leve 
elements in test performance has long been recognized. How- 
ever, in practice it has been assumed that since these element? 
appear to be highly correlated they are merely different aspects 
of the same underlying entity and that consequently the die 
tinction can be ignored. Furthermore, it has been assume 
that in any case the normal exigencies of group test adminis" 
tration preclude any attempt to make separate measurements 
of speed and level. Without undertaking to review the litera- 
ture on the problem, we believe that these assumptions will bear 
analysis. 
Thie assumption that speed and level are different aspects : 
the same thing has arisen partly through confusion in terms ап 
partly through misinterpretation of the experimental evidenc® 
ис most frequent error is that of identifying time-limit oor” 
as "speed" scores and then proceeding to cite correlations Ре 
tween time-limit scores and scores obtained in unlimited time 
The point has been missed that these correlations are spurious A 
high, since they rest on a part-whole relationship. The p 
obtained in unlimited time is equal to the time-limit score pm 
whatever the subject can accomplish in additional time- йт 
over, the correlation between these scores is a function 0 
length of the time-limit, for obviously as the time-lim't 


f the 
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lengthened the time-limit scores become more similar to scores 
obtained in unlimited time. In any case, correlations as high 
as even .7 or .8 are still not high enough to rule out the possi- 
bility that a speed component, independent of level, exists in 
the time-limit scores. 

The number of studies in which correlations have been 
obtained between rate-of-work scores and level scores is exceed- 
ingly small. The obtained correlations are seldom more than 
moderately high but even these have occasionally been cited as 
showing the fundamental identity of speed and level compo- 
nents in test performance. 

These misinterpretations have usually occurred in connec- 
tion with test performances in which the subjects vary con- 
siderably with respect to their ability to answer the items and 
m which the scores involve the number of items correctly 
answered. A particularly dangerous misinterpretation, how- 
ever, is likely to arise in connection with tests in which the sub- 
Jects vary not in their item-passing ability, but only in their 
Tate of performance. One frequently cited study is that carried 
out by Paterson and Tinker (7), who came to the perfectly 
Sound conclusion that when corrected for attenuation, the cor- 
relation between “work-limit” and “time-limit” scores on a 
Speed-of-reading test is virtually perfect. The work-limit score 
Was not the number of items correctly performed in unlimited 
time but, instead, the time taken to read all the paragraphs in 
the test, The time-limit score was the number of paragraphs 
read within a time-limit. What Paterson and Tinker showed, 
then, was that in the measurement of a rate of performance 
it makes little difference whether the scores are expressed in 
terms of performance-per-unit-of-time or time-per-unit-of-per- 
Огтапсе, A convenient paradigm is that of a runner’s speed, 
which can be expressed either in terms of feet per second or in 
terms of seconds per foot. It is a mistake, however, to general- 
ize the results of the Paterson and Tinker study by inferring 
that “work-limit” and “time-limit” scores in the usual sense 
Will be highly correlated in situations where elements of test 
Performance other than rate are measured. 

ith respect to the presumed impracticability of measuring 
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speed and level components separately within a time-limit, we 
can only point out that few attempts have been made to explore 
the problem. With ingenuity, it should be possible to devise 
relatively simple methods of making separate measurements 
even within a reasonable time-limit. 

We would by no means assert that speed and level compo- 
nents of test performance are invariable entities from test 10 
test. Rate of performance in one task may be completely inde- 
pendent of rate of performance in another task. Similarly, level 
components undoubtedly vary from test to test. The investi- 
gation reported here establishes the independence of several 
distinct types of speed, in addition to a general speed facto 
and previous factorial investigations have isolated several types 
of level components (such as vocabulary knowledge, ability t° 
solve problems expressed verbally, etc.). | 

We conclude this general introduction by making several 
recommendations in the fields of test construction and factor 
analysis. First, we suggest that persons responsible for the 
standardization and validation of tests experiment with the 
differential validities of speed and level scores and incorporat® 
any significant findings in the directions for administering, SCOT 
ing, and interpreting the tests. Investigations should be made 
of the possibility of restandardizing various published tests 19 
terms of speed and level. Persons charged with selecting tests 
for use in given situations should give preference to tests whic 
have been so standardized. Collateral experiments sho" 
meanwhile be directed towards discovering more efficient 2? 
reliable methods of measuring speed and level than, say» thos? 
employed in the present investigation. 

Our second major recommendation is that in facto" 
studies aimed at discovering unitary abilities, tests shoul " 
represented by speed scores, level scores, or both, and that ! 
time-limit scores are to be studied at all ghey should be treat? 
in the manner exemplified in the investigation reported here 


jal 


The Experiment 


In order to establish the linear independence of speed Ф 
level scores it was decided to study by factor analysis à mat 


2 
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of correlations between speed, level, and time-limit scores in a 
number of short mental ability tests. As in Baxter's study (1), 
Speed scores were obtained as the number of seconds taken by 
the subject to work from the beginning to the end of the test, 
attempting every item once. Level scores were defined as the 
number of items correctly answered when the subject is allowed 
to take all the time he desires to try every item and to check 
over his work. Time-limit scores were defined as the number 
of items correctly answered within a prescribed time-limit. 

The test battery consisted of the eight subtests of the Re- 
vised Alpha Examination, Form 5; the Minnesota Speed of 
Reading Test for College Students, Form A; and several tests 
which had been specially constructed for previous factorial in- 
Vestigations. These included Letter Grouping and Scattered 
X’s, studied by Thurstone (8); and Phrase Completion and 

isarranged Morphemes, constructed by Carroll (2, 3). The 
Revised Alpha Examination was used because its subtests ap- 
Pear to measure verbal, numerical, and reasoning factors, to 
Judge from Guilford's analysis of the original Army Alpha test 

» and because it is somewhat more practicable to administer 

and Score than the original Army Alpha. Letter Grouping and 

ISarranged Morphemes were included to aid in defining the 

omain of reasoning ability. Scattered X’s was included to test 
the hypothesis that the Perceptual Speed factor (P) as mea- 
Sured by the test might be involved in some of the speed scores 
Studied here, The Minnesota Speed of Reading Test was in- 
cluded because it was believed that speed of reading might be 
Telated to speed scores on mental tests which contain reading 
Material, 

Speed, level, and time-limit scores (as defined above) were 
OPtained for each test or subtest in the battery, with three 
SXCceptions, For the Minnesota Speed of Reading Test, the 
only score obtained was the number of paragraphs marked, cor- 
rectly or incorrectly, in the prescribed 6-minute time-limit. 
dd. WIL. measures the speed aspect of pertormanee on the 
Ed he Score on Scattered X s was the number of X's found 
fire marked in 4 minutes; again, this score is primarily a mea- 

of rate of performance. Phrase Completion had no time- 
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limit and was scored by means of a key the construction of 
which has been described in a previous article (3). Special 
instructions and procedures were devised to obtain speed, level, 
and time-limit scores on the same test. 


A large clock with a sweep-second hand was placed in 
view of the subjects. First the subjects worked on a test 
for the prescribed time-limit, marking an “x” in the margin 
after the last answer written within the time-limit. They 
were then instructed as follows: “Continue working rapidly 
on the test to the end, but do not yet change any answers 
you have already written. As soon as you each individually 
finish the test, quickly look up at the clock and record at the 
bottom of the page the minutes and seconds you required to 
do the remainder of the test below the X. Do not stop too 
long on any one problem. You may guess at answers you 
don’t know or leave blanks. Write the time before you БО 
back to fill blanks or make corrections. After you recor 
your time, you may take your red pencil and make any addi- 
tions or corrections, but do not erase present answers.” The 
students were allowed to work on each test until all had 
finished, except on the Disarranged Morphemes and Letter 
Grouping tests, where a few students were not able to finis 
within 23 and 18 minutes, respectively, after the time-limit. 
This procedure yielded a time-limit score, which resulte 
from the application of the prescribed scoring formula to а 
answers written in ordinary pencil up to the X marked by 
the subject. The speed score was the number of minutes 
and seconds recorded at the bottom of each test. The level 
Score was the score on the entire test; if an answer in black 
was followed by a different one in red, the latter was taken 
as the answer for purposes of arriving at the level score. 


The time-limits used for the subtests of the Revised Alpha 
Examination were those recommended by Tinker and Baker 
(9) for use with college students, For experimental purpose 
scores for two time-limits—2 and 4 minutes —were obtained E 
subtest 2, Arithmetical Reasoning. 

The subjects were undergraduate students in elementary 
experimental psychology at Indiana University. The analys? 
of test scores was based upon 91 complete cases—12 men ар 
79 women. 

The markedly skewed distributions of certain speed scores 
(on the subtests Addition, Common Sense, Same-Opposite, yor 
Disarranged Sentences in the Revised Alpha) were made mor 
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nearly normal by converting them to the reciprocals of the 
number of seconds. Scores on all variables were coded in ten 
or fewer class intervals. Hollerith procedures were used to 
obtain Pearsonian product-moment coefficients. No correc- 
tions for grouping or attenuation were applied to the coeffi- 
cients. Before the correlation matrix was assembled for factor 
analysis, the level and time-limit scores on subtests Addition, 
Common Sense, and Disarranged Sentences were discarded, 
first, because most of the students made perfect scores in un- 
limited time, and second, because time-limit scores were very 
highly correlated with speed scores. Scattered X’s was omitted 
rom the analysis because it was little correlated with any other 
variable in the battery. It was therefore concluded that the 
Perceptual Speed factor as measured by Scattered X’s is not 
Significantly involved in the speed variables studied here. 


The Factor Analysis 


Level and time-limit scores, as defined here, are overlapping 
Measures since the level score on a test can be regarded as equal 
to the time-limit score plus whatever additional correct answers 
the subject can give in time beyond the time-limit. There is 
Kewise an obvious overlap between speed and time-limit scores 
Since the faster the subject works the more items he has an 
°PPortunity to pass within the time-limit. It was believed that 
these factors of overlap would introduce spurious dimensions in 
the factor analysis if the correlation matrix were analyzed in 
the usual fashion. To put the matter differently, insertion of 
the time-limit scores would spuriously raise the communalities 
8 the speed and level scores. A special method of factoring the 
Matrix was suggested by Dr. L. R. Tucker. The main matrix, 
involving only speed and level variables, was analyzed in the 
"sual way by the centroid method. All correlations between 
md Шон scores and speed scores or between time-limit scores 
Bain Scores were placed in a subsidiary matrix and factored 
main tely. (See Table 1 for the correlations represented in the 
"able Subsidiary matrices.) Correlations among time-limit 
involved Were not analyzed at all. Essentially, the procedure 

locating the time-limit variables in the factor space 


EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


418 


uod үешпзәр ayy әуештшцүә оз (уур Aq pardnqnur useeq 
`ў 91991, Ul UAZ әле sajquisea IYI jo soureu IJ, 'stsÁ[eU? 10499) IYJ UI pasn suon?[oi1oo ayy Á[uo squasaid әүдеу SINT, y 


элец $ә1тїцә 1y 


9t Oy OS Sk OF & ol 2 61 OI SE 8f Le #© OF 51 OF OF SF FH 95 OS SH CS Ot 90 SE 
tL, 8t 9t TE 62 tt OF LZ 61 6С vb {Т SC Sh 8t t£ FI 9% SE OF BI 62 9% 9i OZ ZI Sc 
Oc ££ IC #0 61 @ 9 SC OI 6c 6с 40 £I Zi IC OC LO IC FL IL idc 10 IL O0 £O 40 vC 
tf 80 cS te CP 6t Ik 8v St th 62 99 EE SS 9E ҮРӨ] -dy FE BT LE 08 Te SL WS £C 
SP Sp 9 SE I$ Z6 tt Gf 8I Ф 40 9S 6c $9 vt SE £l OF Cb LE St IP SC 05 ST 80 [44 
Of zt £C 6t OF ¿Z @ OF LT SC tl tt 62 ZE vt ҮС SO CI 6C FE LI vC ST OF 2-00 Ic 

Sh LF РС tb ZT SS F9_LE СС IS 81 t£ OS tb SP FE 6C FE OT 9c 0с 


8r SP OF Tr 22 9% 
g € t9 IC SC OF 8$ IZ 9$ FE FE 22 £k £0 ТЕ SC OF “ZI SE СЕ ££ OT £0 8I 


66 Sr Æ SC Sv II- Т2 08 SI SE OZ Ck SE # TS GI IT 9€ Ф 62 FE 6T 81 OF # £C 9T 
OF FI 40. SI (ЕГ “SO SI £0 TC ch 86 9E ££ 6I OC ££ ££ OI 1 


Se OC 8с 85 9% 9I FL 0 
SE 9t 86 Ф OF 92 IC 95 “9% CU Cf TE Ob CF OF 8Е Sc 86 SP OS 9c LO £I 


0Е 09 9S St 
IS 59 95 €9 тр oF 05 £r t£ PL Le Cb GL 95 SC Cb 8C OF — 8S8 49 SP Ik 09 SE Zz zI 
Of II FE LE FE Ch OE 6C 95 8t 85 Ф 6F 65 85 PE SC II 


аа 0 9I 90 cI 40 2-80 00 9% £0 £C OI 40 42 SC 6t s- TE 82 LZ 
РЕ СЕ. t£ OF 82 Le S£ 52 ҮС £t 0С 1С OC 8I 9T 9I Sf CI TI OL 6 8 4 9. S 


хы]рш. „билр, MUD py, 


«722470 Jf w01]7]24407) әү 


I WTISVL 


SPEED AND LEVEL COMPONENTS 419 


defined by the speed and level variables. Factor loadings for 
the variables in the subsidiary matrix were obtained by sum- 
ming the columns of correlations or residuals; the product of 
the column sum and the value (Xr)? used to compute the mth 
factor loadings for the main matrix was the mth factor loading 
for the subsidiary matrix variable. Residuals in the subsidiary 
matrix were computed and treated in the usual way. 


TABLE 2 
The Centroid Matrix 


Tet | I п ш IV M VI т 
5 32 =37 -.10 -18 -.05 27 36 
6 43 -.40 -.09 35 18 -.18 54 
7 71 05 = 4 -.18 =17 EN 75 
8 62 10 -.34 -.16 -.04 23 59 
9 65 12 =28 -.16 -.04 -.05 52 

10 63 -.36 -.10 = 15 10 60 
п 71 -.06 201 -.14 07 -.07 58 
12 76 =21 -.05 -.13 05 -.06 65 
13 64 18 =li 32 -.32 -.02 66 
14 39 i15 - 23 32 -04 = 35 
16 56 -.24 29 19 zd = 10 52 
18 48 4 03 =. 05 32 ~ .04 53 
20 67 =i12 42 -22 =e 205 72 
21 40 18 22 * 20 LET. 36 
22 61 11 33 =17 = 22 -.14 59 

3 65 34 28 16 a8 =з 70 
24 26 06 24 21 21 31 31 
25 51 14 35 ‘05 10 12 43 
25 Бл 17 _ ei 105 22. 5319 56 
2r 69 = 26 22 -.05 = 129 70 
8 58 2105 36 107 02 -.06 55 
20 62 4 - 23 – 20 18 26 77 
F 70 =28 23 25 -.09 -.01 69 
5 71 E - 20 =18 15 SIT 64 
4 75 ‘15 An 03 =01 02 `60 
36 72 26 15 O6 PENES 169 
38 62 13 25 18 17 10 54 
—— 


As shown in Table 2, six centroid factors were extracted. 

€ centroid matrix for the main correlational matrix was the 
asis for the rotation to simple structure, which was accom- 
Plished by Tucker’s semi-analytical method (10) in five trials. 
9 transformation matrix (Table 3) was used to obtain the 
nal rotated matrix (Table 4) both for the main and the sub- 
“бату matrix variables. The time-limit scores did not in any 
Way Influence the rotational procedures; nevertheless, the vec- 
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TABLE 3 
The Transformation Matrix 


A B с р Е A 
42 

I 33 30 16 09 18 
I ES d 0 —-16 m 2 =н 
ш 29 б3 Шш = T an 
IV 04 29 89 -.03 20 aa 15 
у 38 -S4 07 74 s -—h 

VI 27 0 -30 -6 79 


8 Р eu 
tors for these scores were found to fit well in the simple p 
ture already established by the speed and level scores. Table 
shows the correlations between the primary factors. 


TABLE 4 
The Rotated Factorial Matrix 
Test Variable 4 B р D d ^ 
Speed Scores: 


4 
Addition .................. 38 =f =n ый 4 $ 


5 

6 

ГА 

8 Same-Opposite .. 
9 Disarr. Sentences 
10 Number Series .. 
11 Verbal Analogies 
12 Directions .... 
13 Disarr. Morphe 
14 Letter Grouping .. 


Level Scores: 
16 Arith. Reasoning 
18 Same-Opposite ... 


.- 05 әй 3 2 v 
20 Number Series ..... "s 53 -16 - М -03 = 10 
21 Verbal Analogies .... = 02 4 -.1 30 -02 Е: 
22 Directions ........... sno d0 SL 10 04 “б 


23 Disarr. Morphemes ... 
24 Phrase Completion 
25 Letter Grouping 


35 Speed of Reading 


Time-Limit Scores: 


06 
27 Arith. Reasoning .. (2^) ... 43 46 32 08 -.06 08 
28 Arith. Reasoning .. (4^) ... .50 46 10 —.02 04 46 
30 Same-Opposite ... (1)... —08 -13 -16 4 46 “yg 
32 Number Series ... (24’) .. 48 34 —13 -.08 =.04 42 
33 Verbal Analogies .. (2^) ... .30 -09 05 30 - (e 38 
34 Directions ....... (7) ... 10 18 16  .10 15 11 
36 шап. Morphemes (8) ... .04 44 28 08 37 02 
38 Letter Grouping .. (8) ... 24 36 17 07 
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TABLE 5 


Correlations between the Primary Factors 


A B C D E F 

4 1.000 —.044 -.082 -.035 -242 .002 
B -.044 1.000 -.422 688 052 635 
C -.082 -422 1.000 — 460 .051 = 401 
р -.035 .688 - 460 1.000 .130 516 
E -.242 .052 .051 131 1.000 -.036 
F .002 635 -.401 516 — .036 1.000 


Interpretations of the Factors 


In interpreting the factors we follow the arbitrary rule that 
à projection larger than .30 indicates a significant loading of a 
test on a factor. 

The variables having projections of .30 or greater on factor 
A are ranked below in order of size of projection. Significant 
Projections on other factors are also given. 


Projections 


No. Variabl 
n тага А Other factors 
10 Number Series (speed) ..........:.+. 53 AIF 
28 Arithmetical Reasoning (4^ time-limit) .. .50 46B 
32 Number Series (time-limit) ws 48 34B 
6 Arithmetical Reasoning (speed) ........ 46 54C 
27 Arithmetical Reasoning (2^ time-limit) .. 43 6B; .32C 


5 Addition (speed) ........ 38 34F 
12 Directions (speed) ..... " 38 .36F 


16 Arithmetical Reasoning ( 38 50B 
20 Number Series (level) .... 31 .53B - 
33 Verbal Analogies (time-lim .30 30D; .4 


Most of these tests obviously have to do with simple arith- 
Metical computation; factor A, then, appears to be the number 
factor N identified in previous studies. The speed scores of 
Arithmetical Reasoning and Number Series have higher load- 
Ings than the corresponding level scores. The interpretation 
may be offered that for college adult subjects this factor refers 
to the speed aspect of computational behavior. The level of 
competence of these subjects is such that accuracy in arithmetic 
Plays only an incidental róle, although rapid arithmetical abil- 
Пу appears to facilitate the correct performance of the rela- 
tively complicated tasks set in Arithmetical Reasoning and 

umber Series. The presence of Directions (speed) on this 
factor becomes understandable when it is noted that a con- 
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siderable share of the items involve numbers and numerical 
operations. 
Factor B has the following tests: 


B Other 

23 61 — 

20 53 31A 

22 51 A 

16 50 38A 

27 46 434; .32C 
28 Arithmetical Reasoning (4^ time-limit) .. 46 .504; .10C 
36 Disarranged Morphemes (time-limit) ... 44 

13 Disarranged Morphemes (speed) ....... 39 34C; .36Ё 
25 Letter Grouping (level) ES .38 31E 

38 i 36 37E 

22 34 48А 


In previous factorial studies tests similar to those represented 
above have been identified as tests of reasoning ability. Of 
the eleven variables listed, ten are directly or indirectly mea- 
sures of level of ability (time-limit scores being regarded as 4 
function of both level and speed). In the light of these con- 
siderations, factor B may be identified as a Level of Reasoning 
factor. The present battery is too limited to indicate the vale 
tion of this factor to the inductive and deductive reasoning 
factors which have been indicated in previous studies. The 
presence of thé speed score of Disarranged Morphemes on Шш 
factor is interesting. In contrast to other tests in this battery» 
Disarranged Morphemes is of such a nature that it is almost 
impossible for a subject to be satisfied with an incorrect answe? 
the subject either solves an item correctly or is forced to skip 1 
Consequently, speed of performance in this task would Þe 
almost perfectly related to ability to answer the items if the 
subjects did not differ.in their willingness to skip items. Ps 
cause of this inherent connection between speed and level 
aspects of performance on the Disarranged Morphemes test, 
1s not surprising to find the speed score present on the Level ? 
Reasoning factor. Parenthetically, we may say that there are 
several subtle problems in this area which this study has no 
been designed to handle. For example, one would like to kno 
how the speed-level relationship varies with the difficulty of tae 
task and whether the relationship in the case of multiple-cho!c* 
tests is essentially different from that in the case of tests Whe! 
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the subject is forced by the nature of the test to answer cor- 
rectly or not at all. 
Factor C is represented by the following test variables: 


C Other 
6 Arithmetical Reasoning (speed) ....... .54 46А 
14 Letter Grouping (speed) ....... А 46 ЗАР 
13 Disarranged Morphemes (speed) ...... 34 39B; .36F 
27 Arithmetical Reasoning (2^ time-lim Doy .32 АЗА; 46B 


The tests represented here are reasoning tests also found in 
factor B. Factor C, however, is constituted by measures of 
Speed. Level is not independently represented at all. Factor 
C may hence be regarded as a Speed of Reasoning factor. As 
will be shown later by multiple regression techniques, the time- 
limit scores of these reasoning tests are much more heavily 
weighted with level than with speed. It is not surprising that 
only one time-limit score (from the 2’ time-limit on Arithmeti- 
cal Reasoning) appears on factor C. The 4’ time-limit score 
on this test has a loading of only .10 on С. 

It is of interest to note from Table 5 that there is an appreci- 
able negative correlation between factors B and C. This proba- 
bly indicates that when other factors are ruled out, those who 
are hasty in performing these reasoning tests are likely to be 
Inaccurate, 

Factors D and £ lack definition in the present limited bat- 
tery. They are represented by the following variables: 


Factor D: D Other 
35 Speed of Reading .................. .38 A6F 
Same-Opposite (level) .. 34 .32Е 
21 Verbal Analogies (level) mue .30 —— 
33 Verbal Analogies (time-limit) ....... .30 1304; 42F 
Factor Е: Е Other 
24 Phrase Completion ................. 46 — 
Same-Opposite (time-limit) ......... 46 A6F 
38 Letter Grouping (time-limit) ........ 37 36B 
Same-Opposite (level) .............. 32 34D 
Letter Grouping (level) ............ Al 38B 


Factor D may perhaps be characterized as a verbal reasoning 
factor which emphasizes formal relationships such as those of 
antonymity, genus-species, etc. Factor D is highly correlated 
with factor B, the Level of Reasoning factor. Were it not for 
the presence of both the level and time-limit scores of Letter 
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Grouping on the factor, factor Ё might readily be interpreted 
as the verbal factor identified in previous studies. 
Factor F is represented by the following variables: 


F Other 
7 Common Sense (speed) .............. 63 — 
8 Same-Opposite (speed) ............... .62 — 
11 Verbal Analogies (speed) ............. 46 — 
35 Speed of Reading .................... 46 38D 
30 Same-Opposite (time-limit) ........... 46 AGE 
9 Disarranged Sentences (speed) ........ 43 — 
33 Verbal Analogies (time-limit) ......... 42 304; .30D 
10 Number Series (speed) ............... AL .534 
34 Directions (time-limit) ............... .38 Sa 
12 Directions (speed) ......... eee 36 38A 
13 Disarranged Morphemes (speed) ...... 36 39B; 34 
5 Addition (speed) .................... 34 .384 


14 Letter Grouping (speed) .............. 34 46C 


Every one of these variables involves either a direct or an indi- 
rect measure of speed. It is also true that with one exception 
every speed score in the battery appears in the above list. Only 
four time-limit scores are absent: those of Arithmetical Rea- 
soning, Number Series, Disarranged Morphemes, and Letter 
Grouping, and in these tests it can be shown that speed СОЛ" 
tributes little to the time-limit scores. Hence it may be СОЛ" 
cluded that factor F is a general speed factor involving rate ° 
work in performance of tasks of the sort found in intelligence 
tests. The factor is similar to a general speed factor found 19 
some of Holzinger’s studies (5,6). The content of a test does 
not seem to play any róle in determining the loading of its spee 
score on factor Ё, since tests of verbal, numerical, and reasoning 
abilities all appear in the above list. No definite conclusions 
can be drawn from the present data, however, as to whether this 
factor extends to both easy and difficult tasks. 

The presence of Speed of Reading on factor F might lead 
one to suspect that speed of reading is fundamentally involve 
in this factor. However, some of the tests whose speed scores 
measure the factor (e.g., Addition, Number Series, and Same 
Opposites) do not have items containing connected text-mato" 
rial where a speed of reading factor could be expected to oper. 
ate. It appears, therefore, that an individual's reading spe? 
is partly a function of some more general speed factor. 


оы-токекгп47 
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Multiple-Correlation Analysis 


Although it is believed that factorial techniques provide a 
more complete and concise analysis of the data, multiple-corre- 
lation techniques can be used to evaluate the independent róles 
of speed and level in determining the variance of time-limit 
scores. Table 6 presents such an analysis for all tests in the 


TABLE 6 


Beta-Coefficients and Multiple Correlations in the Prediction of Time-Limit 
Scores (T) from Speed (S) and Level (L) Scores 


Relative 
Contributions 


Zero-order 
Correlations 


Test Кт.ви 


Alpha Examination: 


l. Addition ....secsisioaseasi à 4 А 1 s .826 
2. Arith. Reas. (2^) 5 811 
дъ 76 “ (47) 711 
3. Common Sense . :843 
4. Same-Opposites ....... А 835 
5. Disarranged Sentences .... 842 
6. Number Series ........... -840 
7. Verbal Analogies ......... -840 
8. Directions .............-- 724 
Disarranged Morphemes ....... TE 


battery for which speed, level, and time-limit scores were ob- 
tained. The beta-coefficients indicate the relative contribu- 
tions made by speed and level in predicting time-limit scores. 
In some tests, such as Arithmetical Reasoning, the time-limit 
Scores are chiefly a function of the level of item difficulty that 
can be mastered by the subject, while in other tests, such as 
Common Sense, the time-limit scores are primarily measures of 
the subject’s rate of work. In still other tests, such as Same- 
)pposite, speed and level are about equally weighted in the 
time-limit score. These relationships depend to some extent 
9n the particular time-limits which had been set. 

Even where the correlation between time-limit and level 
Scores is fairly high the contribution of an independent speed 
Component to the time-limit score is sometimes fairly large 
(e.g., in the case of Letter Grouping). The multiple correla- 
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tions in Table 6 are the correlations obtained in the prediction 
of time-limit scores from level and speed scores. These multi- 
ple correlations are in some cases considerably higher than the 
corresponding zero-order coefficients. Nevertheless, there re- 
mains in each case a certain amount of specific (unpredicted) 
variance in the time-limit score which would militate against 
the prediction of level scores, for example, from a weighted 
combination of speed and time-limit scores. 


Summary 


A number of relatively simple group mental tests мее 
administered to 91 college students in such а way as to yiel 
three types of score: speed, level, and time-limit. Speed scores 
represented the time required to attempt every item once; level 
Scores represented the number of items correctly answered in 
unlimited time; and time-limit scores were the number of items 
correctly answered in a prescribed time-limit. Factor analys! 
revealed that in all cases speed scores were linearly independent 
of level scores and that time-limit scores could be represente 
as factorially complex measures having loadings on both spee 
and level dimensions of ability. Of the factors which were 
identified several were similar to verbal, numerical, and reason" 
ing factors isolated in previous factorial studies. In the doma? 
of reasoning ability both level and speed factors were identified- 
A general speed factor involving nearly all of the speed scores 
was found. It is concludéd that because of their factorial com- 
plexity, time-limit scores should be used with considerable 
caution both in factorial studies and in studies involving th® 
prediction of criteria. 
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THE USE OF AN OBJECTIVE TEST IN PREDICTING 
RHETORIC GRADES 


IRWIN A. BERG, GRAHAM JOHNSON, лхо ROBERT P. LARSEN 


University of Illinois 


Wuite a passing grade in rhetoric or in an equivalent course 
is required of freshmen in virtually all colleges and universities, 
many institutions are exempting from the course those students 
Who pass a proficiency examination. In addition, such pro- 
ficiency examinations are sometimes used to determine whether 
a student should be admitted to the regular course in rhetoric 
or assigned to a special non-credit rhetoric class. 

At the University of Illinois all entering freshmen are re- 
quired to take a rhetoric placement examination. Those stu- 
dents whose performance is high are granted credit in Rhetoric 
1 without taking the course. Those whose test performance is 
low are required to take Rhetoric 0, a non-credit course. Pro- 
vision is made for early transfer to Ересек 1 of any Rhetoric 0 
students whose classroom work proves to be at the level found 
in Rhetoric 1. All other students are entered in Rhetoric 1, the 
usual college course. In addition, a recent action of the Uni- 
Versity's Board of Trustees makes reasonable proficiency in 
written English a graduation requirement. Students earning 
grades of “С” or “D” in Rhetoric 2 are required to pass a special 
examination or to pass a third course in Rhetoric before being 
granted a bachelor's degree. 

The Rhetoric Placement Examinations at the time of this 
Study in October, 1943, consisted of an objective test’ and an 
Impromptu theme written in the examination room. The 
actual decision as to whether students were assigned to Rheto- 
тїс 0 or Rhetoric 1 or passed for proficiency was made on the 


basis of the quality of the impromptu theme. Two rhetoric 
Á 
1 Cooperative Test A: Mechanics of Expression, Form Q. 
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staff members individually evaluated each theme for — 
to one of the three groups. If the two instructors disagree i 
the theme was given to a third instructor who made the fina 
decision. The third instructor could consult the objective test 
score in such contested cases when the objective test results 
were available. The objective tests were scored by an Inter- 
national Business Machines electrical scoring machine. | 
The object of this study is to make inquiry into the useful- 
ness of an objective test in predicting rhetoric achievement 1n 
college. Upon initial examination it may appear that such Я 
inquiry could not prove fruitful since the rhetoric course gr 
is mainly determined by grades on what is often thought of as 
` subjectively evaluated compositions. Thus a sample comp? 
sition graded in the same manner might be considered an emi- 


г s Wb ois Е п 
nently more suitable predictive instrument. As Kelley а 
Roberts put it: 


“We have found that ability to detect and correct errors 
in exercises is not always accompanied by the ability to 
avoid similar or worse errors in original composition, an 
that, conversely, students really proficient in composition 
may have indifferent success on a problem-solving type О 
test. We hold firmly the conviction that a student's degree 
of proficiency in writing can be determined only by a demon- 
stration of that proficiency, in writing"? 

P 2 5 b 

Yet the advantages of rapid scoring which could be дак 
persons who are not necessarily rhetoric instructors, um 
with the advantages of objectivity of score, would make Xd 
use of a suitable objective test an extremely practical meas 
ing tool. red 

Two groups were used in this study. Group 1 number, 
372 students and group 2 numbered 166. Both groups ж. 
composed of freshmen entering the College of Liberal Arts e 
Sciences in the fall of 1943. The groups were not at first get 
bined because each was tested in a different room and by di 
ent examiners, although the day and hour were the sanp d 
both groups. As will be noted later this precaution was 


28 
necessary; consequently only in Table 2 are the groups ёр 


- 


; „ llino” 
? Kelley, Cornelia and Roberts, Charles. “Rhetoric Proficiency Tests. 4 
English Bulletin, ХХХІ (1944), 2. 
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rated. In analyzing the data from these groups Pearson prod- 
uct-moment correlations and other simple statistical measures 
were used. The grades of Rhetoric 0 students were lowered one 
grade point (i.e., from B to С) and the grades of students who 
Were passed for proficiency in rhetoric were tabulated as B. 
This was in accordance with the recommendation made by 
members of the rhetoric staff. 


Results 
TABLE 1 
Mechanics of Expression Test Scores in Relation to Grades in Rhetoric 
Rhet. 
Grade N Mean S.D. С.К. 
tesa 25 142.7 9.8 
B eene 154 1176 74. 1050088) 
MELLE 16 98.4 202 
i Mese pe E: 831 202 46(C&D) 
Ас ү сены ds 8 73.3 29.5 9 (D&E) 
d 
Rhetoric 0 Rhet т E ЕН 1054 163 3.2 (D & Rhet. 0) 
Pass Proficiency .... 62 134.5 15.3 3.0 (A & Prof.) 
TABLE 2 
Correlation of Grades with Test Scores 
Group 1 Group 2 
N г N m 
Ме. Ex 696 
* &Xp. Scores and all Rhetori des* .. 372 .683 166 К 
Ме. Exp. Scores and Rhet. 1 grades onlyt .. 350 60 159 1634 
prog: P: Scores and Rhet. 1 grades without 5 
Mech Siency groupt ...................- 618 13, 3 
» Exp, Л 
Май course and nde Pon ae ee ME 
AC} САР. Scores and ACE. Total Scores|| .. 342 .659 E 
CE. Total Scores and ЕТ кайы 302 442 139 465 


. 
Rheto grades of students “passed for proficiency” in Rhetoric 1 are taken as SRS 
Coopera; Ü grades are lowered one grade point. Mech. Exp. is the abbreviation for 
tive Test A: Mechanics of Expression, Form Q. 
t Rass 0 grades are not included. y 
5 оне 0 апа “passed for proficiency” grades not included. 
hours У those students who earned grades in courses totaling 12 or more semester 
аге included. 
tion, 1946. refers to the American Council on Education Psychological Examina- 
» 1940 edition. 


the Ae 13296 0 grades аге not included. Some Rhetoric 1 students did not take 


^. examination. 
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TABLE 3 


Types of Errors Checked in Early Freshman Compositions of 147 
Rhetoric I Students* 
——— ЕЁ 


Percentage of 


Type of Error Total No. of ^ Students Making 


Violations One or More 
Errors 
1. Grammar (i.e., sentence fragment, incorrect 
tense or mood) ai 364 33.9 
2. Mechanics (i.e., capitalization, i GE 298 44.1 
3. Punctuation (ie, superfluous comma, 
hyphen in compounds) . 1153 50.2 
4. Spelling .................. 703 86.0 
5. Diction (ie, exactness, wordiness, 
Ае Зноу одн 1184 75.9 
6. Coherence (i.e., word order, dangling modi- 
fiers; parallelism): „әзән. 636 39.7 


* Summarized from Johnson, W. G. and Mathews, E. G. Errors most frequen 
баа in early freshman compositions. Ilinois English Bulletin, ХХХІ ( ч 


Discussion 


The curious aspect of these data is not that the objective 
test used in this study is a good predictor of achievement i" 
Rhetoric 1. Admittedly, correlations above .60 between course 
grades and pre-test scores are not common. The more pert! 
nent question is why is the correlation so high? Rhetoric Е 
a course in which the final grade is largely determined by grades 
earned on written themes in which the instructor’s evaluation" 
includes what may be considered subjective aspects such as 
triteness, wordiness, or lack of logic. The purely objective test, 
on the other hand, measures only basic skills in grammars 
spelling, punctuation, and capitalization. There can be little 
doubt that the objective test does bear a clear relation to grades 
earned. In Table 1 it will be seen that there is a progressiv? 
decrease in the mean objective test scores by grades from A 
through “E.” Rhetoric 0, composed of students deemed Wo 
poorly prepared to enter Rhetoric 1, has the lowest mean of a 
categories. The group “passed for proficiency” has a mean 
score between those of the “A” and “B” grade students. Sev- 
eral Rhetoric 1 instructors agreed beforehand that the pre 
ficiency students would probably fall at this level. The critic 
ratios of the differences between the test means of most 814 
categories indicate that the differences are highly significant- 
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An inference may be drawn from the data in Table 3 which 
partially explains why the predictive value of an objective test 
should be high for a course the content of which is ordinarily 
thought of as being subjectively graded. It will be noted that 
the majority of the errors of early freshman compositions were 
largely objectively ascertained errors, i.e., errors of spelling, 
Punctuation, mechanics, and grammar. Significantly, it is this 
type of error which the objective test measures. In practice, 
then, the majority of the errors checked are detected objec- 
tively. 

The weight, or value, rhetoric instructors assign to purely 
objective errors is probably quite high. A hypothetical case 
may demonstrate the importance of the value assigned to a 
given error. Suppose that two students each have exactly the 
Same number of errors checked on their themes. The first stu- 
dent makes errors only of exactness and wordiness while the 
errors of the second student are errors in spelling, such as 
alright, and a series of errors such as between him and I, he 
come around the corner, etc. It is likely that the rhetoric 
Instructor would consider the latter errors as abominations not 
to be lightly dismissed. Presumably the grade of the first stu- 

ent would be significantly higher than that of the second 
although the number of errors was the same in either case. 

In other instances the instructor may grade compositions on 
а purely objective basis because he can do little else. Most 
9! the themes may, at times, approximate each other in errors 
of diction, coherence, and organization while the variation be- 
tween compositions in spelling, punctuation, etc., is marked. 

So, the instructor may be influenced by the fact that such 
“trors are more easily detected and are less likely to be con- 
tested by the student. 

nother interpretation of these data might propose that 
there exists a parallel development of the mastery of English 
mechanics and of effectiveness in expression. Thus, skill in 
mechanics would tend strongly to be associated with variety 
lorian structure, freshness of treatment, and superior 
tend Nu Conversely, lack of mechanical accuracy would 
ngly to be associated with a less effective presentation 


of material. It might then be said that rhetoric instructors are 
accurate in evaluating both the mechanics and the effectiveness 
of composition. Since both aspects of writing are positively 
correlated, a false impression is given that the purely objective 
errors are emphasized. This interpretation would be strength- 
ened by the fact that early freshman compositions, such as 
those recorded in Table 3, are very carefully scored for mechani- 
cal errors. Because most students would presumably improve ; 
in mechanics, the rhetoric instructor could place greater empha- 
sis, when scoring later compositions, upon interest, origina 
and freshness of treatment, ‘This explanation would probab y 
be favored by rhetoric staff members. 

But it must be emphasized that, while such parallel develop- 
ment may exist to some extent, instructors would agree closely 
on scoring for mechanical accuracy but not on scoring for Mt 
nality or superiority in organization. Yet the correlation be 
tween a test of basic mechanical skills given at the beginning 
of the semester with final rhetoric course grades at the end o 
the semester is almost .70. It would seem that if the more 07 
less subjectively scored aspects of compositions are given fane 
weight the correlation should be considerably lower, since var! 
bility in scoring is greater. ; 

In general, it may be said that the high-school prepara j 
of many students is inadequate insofar as mastery of basic ski 

b 
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in the mechanics of expression is concerned. It may be ae 
universal college admission requirement of four years of hig i 
school English would improve the situation. Perhaps 2 E 
quired level of proficiency on a rhetoric test could be made p^" 
of the admission procedure. Such measures could be adopt? 
only if it were agreed that the main function of a college rhet? 
ric course is practice in English composition, not drill in pune y 
ating such compositions. It is probable that the predict! 

value of an objective test in rhetoric will remain high as pi 
as many students enter college with inadequate grounding d 
English mechanics. The correlation of such tests with 81а Ee 
should become progressively lower as students enter rhet? 

classes with more thorough preparation in English. 


» 
DET : :ects 
This lack of thorough training in so-called “drill subje¢ 
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may be reflected in other areas. It is not uncommon to en- 
counter students, for example, who do not know the multipli- 
cation table. Occasionally one may discover a college student 
who has not learned the exact order of the letters of the alpha- 
bet. Whether or not mastery of “drill subjects” should be 
expected of entering college students cannot be discussed here. 
But it seems clear, in the case of mechanics of English at least, 
that such lack of proficiency does exist. 


Summary 


Data from 538 University of Illinois Liberal Arts College 
freshmen in Rhetoric 1 were analyzed. It was found that the 
Scores of an objective test in the mechanics of expression corre- 
lated as high as .69 with final grades in rhetoric. Critical ratios 
of the differences in the mean objective test scores of students 
€arning final grades of “A,” “B,” “C,” etc., were significant. 

€ question was raised as to why course grades which were 
Presumably determined subjectively should correlate highly 
With scores on an objective test. The explanation advanced 
Was that the preparation of many students was such that the 
rhetoric instructor was forced to grade largely on the basis of 
errors in mechanics. It was further suggested that instructors 
Probably view more sternly objectively ascertained errors such 
as he dowt and alright than the more subjectively determined 
errors such as triteness and freshness of treatment. Also, it 
Was considered possible that the instructor may have been in- 
fluenced when grading English compositions by the fact that 
Purely objective errors are more easily detected and that the 
Student is less likely to contest a grade based upon objective 
errors, The possibility that a parallel development exists 

tween mastery of English mechanics and effectiveness of ex- 


Pression was considered but judged to be inadequate as an 
explanation. 


A QUICK GRAPHIC METHOD FOR PRODUCT 
MOMENT ‘г 


WILLIAM LEROY JENKINS 
Lehigh University? 


Tu product-moment coefficient of correlation can be deter- 
mined graphically in a fraction of the time required to compute 
it mathematically. With large samples the two methods give 
virtually identical results. With small samples graphically 
determined coefficients appear to be about as representative of 
the relationship in the total population from which the sample 
Was drawn. 

The graphic method can be applied directly to raw data 
Without grouping into class intervals. The correctness of the 
solution can be readily checked by inspection. 

The method depends on the following relation between the 
coefficient of correlation (7) and a ratio (J) which can be deter- 
mined graphically: 


Procedure (See Figure 1) 


0. Plot the scattergraph directly from the raw data without 
grouping. 

1. Move a straightedge from the top of the scattergraph, 
keeping it parallel to the x-axis until 16% of the plotted points 
show above the straightedge. Through the latest point to 
appear draw a line parallel to the x-axis. 

2. Move a straightedge from the right side of the scatter- 


graph, keeping it parallel to the y-axis, until 16% of the plotted 
es 


1 Оп leave until J 1, 1946, with Columbia Universi ivisi 
Re until January 1, 1946, wit umbia University, Division of War 
ee Training Section, Box 34, Submarine Base, New London, 
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points show to the right of the straightedge. Through the latest 
point to appear draw a line parallel to the y-axis. 

3. Move a straightedge from the bottom of the scatter- 
graph, keeping it parallel to the x-axis, until 16% of the plotted 
points show below the straightedge. Through the latest point 
to appear draw a line parallel to the x-axis. 


Fic. 1. 

4. Move a straightedge from the left side of the scatte"; 
graph, keeping it parallel to the y-axis, until 16% of the plott? 
points show to the left of the straightedge. Through the lates* 
point to appear draw a line parallel to the y-axis. ; | 

5, 6. Draw the diagonals of the rectangle formed by раб 
1, 2, 3, 4. é 

7. Move a straightedge from the upper right corner of th 
scattergraph, keeping it parallel to diagonal 5, until 8% of t 
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plotted points show beyond the straightedge. Through the 
latest point to appear draw a line parallel to diagonal 5. Move 
the straightedge in until 16% of the plotted points show beyond 
the straightedge. Through the latest point to appear draw a 
line parallel to diagonal 5. 

8. Move a straightedge from the lower right corner of the 
scattergraph, keeping it parallel to diagonal 6, until 8% of 
the plotted points show beyond the straightedge. Through the 
latest point to appear draw a line parallel to diagonal 6. Move 
the straightedge in until 16% of the plotted points show beyond 
the straightedge. Through the latest point to appear draw a 
line parallel to diagonal 6. 

9. Move a straightedge from the lower left corner of the 
Scattergraph, keeping it parallel to diagonal 5, until 8% of 
the plotted points show beyond the straightedge. Through 
the latest point to appear draw a line parallel to diagonal 5. 
Move the straightedge in until 16% of the plotted points show 
beyond the straightedge. Through the latest point to appear 
draw a line parallel to diagonal 5. 

10. Move a straightedge from the upper left corner of the 
Scattergraph, keeping it parallel to diagonal 6, until 8% of 
the plotted points show beyond the straightedge. Through 
the latest point to appear draw a line parallel to diagonal 6. 
Move the straightedge in until 16% of the plotted points show 


TABLE 1 
Conversion of J to т 
a ee эь 


Ј е gt Гат st ye) ut Л -x* 
l0 .000 .000 2.3 682 .833 3.6 .857 1.281 49 920 1.589 
11 .095 .095 2.4 704 .876 3.7 .864 1.308 5.0 .923 1.609 
12 .180 182 2.5°.725 .916 3.8 .870 1.335 51 926 1.629 
13 256 262 2.6 .743 .956 3.9 .877 1.361 5.2 .929 1.649 
l4 324 .337 24 159 .993 40 .883 1.386 5.3 .931 1.668 
1.5 385 406 2.8 .774 1.030 41 .888 1411 5.4 .934 1.686 
l6 .438 470 2.9 .788 1.065 42 .893 1.435 5.5 .936 1.705 
1.7 486 .531 3.0 .800 1.099 43 .898 1.459 5.6 .938 1.723 
18 .529 588 S.L 811. 41.131 44 902 1482 5.7 .940 1.741 
l9 56 642 3.2 .822 1.163 4.5 .906 1.504 5.8 .942 1.758 
2.0 .600 .693 3.3 .832 1.194 46 .910 1.526 5.9 .944 1.775 
2 .630 .742 3.4 .841 1.224 4.7 .913 1.548 6.0 .946 1.792 
22 658 789 3.5 .849 1.253 48 .917 1.569 


* See R. A. Fisher's “Statistical Methods for Research Workers," pp. 202-215. 
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beyond the straightedge. Through the latest point to appear 
draw a line parallel to diagonal 6. 

11. With a millimeter scale measure the sides of the paral- 
lelogram formed by lines 7’, 8, 9, 10°. Divide the longer by the 
shorter side to get the ratio J’. 

12. With a millimeter scale measure the sides of the paral- 
lelogram formed by lines 7”, 8", 9", 10”. Divide the longer by 
the shorter side to get the ratio J”. . 

13. Take the mean of J’ and J” to get J. Interpolate 1n 
Table 1 to get the value of r or use the equation: 


Mathematical Proof (See Figure 2) 


Commander Н. $. Sharp, 
matical support for what was 


P e- 
USCG, has been good enough to supply this math 
originally a purely empirical method. 


Fic. 2. 


" . d 
Lines 5 and 6 are located by estimating 202 and 2oy (68% of the points) ар 


" ; :ned bY 
drawing the diagonals. Therefore, tan a=, The ratio J’ is similarly obtained : 


; à oe Beni 
estimating the standard deviations (os, ов) about lines 5 and б. Therefore, J^ ^ ce 


1 
of = N эаг=\х (=) sin? a 
zx? 2 
Е Gi t 


Е (o + 


E 


Zxy. бв | Ху? .924X oy 
N o М ар) ox toy? 


Уху a=) oy 


2 totor’ 
N ozoy V oy] os toy 


2 ozo 
or toy 


=(1+r) 
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вё= y ®аё= y x (y- Hx) cos а 
ру 2 ORS 
aa ro 
Oe ite "| E 
Sea Tag ia l-r "=J 


" 
(В. A. Fisher's) Z=4 loge lir =loge J’ 


(The use of J” (8%) has been added to the original method because the mean 
of J’ and J” empirically gives better results than J’ alone.) 


Empirical Tests 
A symmetrical correlation array of 1000 pairs was developed 
from two normal distributions. For this, the computed and 
graphic coefficients were virtually identical. 


Computedr .764 
Graphic r .763 


The pairs were thoroughly shuffed and dealt out into packs 
of 50 each. Each pack of 50 pairs was plotted on a separate 
scattergraph and the coefficient of correlation determined 


COMPUTED 50'S 


d 
cb 


| 
ҳу х x 
ip c EN i i 
x X 
<b KIQ x XX 
40 50 .60 T .80 9 


[*] 


COMPUTED 100'5 


Fic. 3. 
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graphically and by computation. The pairs were gene? 
oughly shuffled and dealt out into packs of 50 each, "s = 
graphic and computed correlations determined аы - 
The same process was repeated to obtain 20 sets of 100 ea т ^ 

Figure 3 shows the frequency distributions of мст iim 
graphic coefficients for the forty sets of 50 pairs and the tv =e 
sets of 100 pairs. Except for one low erratic value the p 
and the computed distributions are comparable. Table 2 s 


TABLE 2 
Standard Deviations with One Erratic Value Omitted 


50 pairs 100 pairs 
2 
peda Е rod pr 
‘computed .... # 
Graphic 064 040 


the standard deviations measured from the ©? of the ae 
population with one low erratic value omitted. The BD ally 
deviations for graphic and computed values are virtu 
identical. ; 
In the original use of the graphic method only the i 
formed by counting in 16% toward the diagonals were ые? 
In these tests 8% and 24% were also tried. Table 3 shows СО 


TABLE 3 


Comparative Standard Deviations 


50 pairs 100 pairs 
Mean 8% and 16% .......... 064 00 
1675, BON: ,— 22:5 eror eine .079 e 
Mean 8%, 16%, and ye .067 x 


parative standard deviations for 
875 and 16% ratios, 
ratios. The describe 
is clearly better than 
8%, 16%, and 24%, 

he standard de 
and compute 
for the 100- 


A о 
16% alone, for the ee 
and for the mean of 8%, 16%, and 


d method (mean of 8% and 16% o 
16% alone and is as good as the m . 

which requires much additional we nic 
viation of the differences betwee? е 25 
d values is 37 for the 50-pair coefficients 229 jas 
Pair coefficients. This is less than the $ 


"че 


‚а 
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deviations of the coefficients themselves, showing that graphic 
and computed values were very much alike in the arrays used 
in these tests. 

There is one type of scattergraph, however, where the 
graphic coefficient is bound to be quite different. That is the 
scattergraph having a generally symmetrical array but with a 
few wildly deviant cases off in one corner. In such a case, the 
computed coefficient may be greatly disturbed by the few devi- 
ant cases. The graphic coefficient will not be affected. In this 
instance, the graphic coefficient probably yields a better mea- 
sure of the true relationship in the whole population from which 
the sample was drawn. 


MEASUREMENT NEWS* 


Papers relating to the field of measurement constituted more than 
half of those presented at the meeting of the Military Division of the 
American Psychological Association at the University of Maryland 
on November 27 and 28. The program included the following papers 

irectly concerned with the field: 
“Equivalences between Army and Civilian Tests”, С. Р. Sparks. 
he Army General Classification Test”, Staff, Personnel Re- 
Search Section, Classification and Replacement Branch, AGO, read 
У R. H. Bittner. 

“Correction for Restricted Range”, E. С. Brundage. 

“The Objective Measurement of Flying Skill”, A. C. Tucker. 

“The Selection of Marine Officer Candidate", S. B. Williams. 

"Surveys of Opinions on Training", C. R. Pace and D. L. Gibson. 

“Scale and Intensity Analysis for Attitude, Opinion and Achieve- 
ment", Louis Guttman. 

"The Form of Items and the Distributions of False Positive Scores 
9n a Neurotic Inventory", W. A. Owens. 

"War Weariness and Morale in Air Groups", J. G. Darley. 

orale Surveys in the Army", Carl Hovland. 

"The Criterion in Army Personnel Research", Staff, Personnel 
VE pecu, Classification and Replacement Branch, AGO, read 
У £. D. Sisson. 

= he Nominating Technique”, C. L. Vaughn. ‘ 

‘The Use of Order-of-Merit Rankings as a Criterion of Shipboard 
Performance of Enlisted Personnel”, Н. Р. Bechtoldt, Р. В. Stuit, 


and J. W. Haucker. 


w ‘Criteria of Air Crew Proficiency in Operational Training”, L. B. 
ard. 


‘Selection of Army Officers”, M. W. Richardson. 
N € Significance of Case History Items as Detectors of Potential 
aval Delinquent”, Н. F. Hunt and Nathan Goldman. 
Su; Ssessment of the Whole Person: Procedures used in Testing the 
ultability of 5,500 OSS Recruits”, H. A. Murray. 
Test Procedures for the Psychiatric Screening of Naval Per- 
sonnel: Some Problems in Method”, Milton Wexler. 
P evelopment of an Interview for Selection Purposes”, Staff, 
“tsonnel Research Section, Classification and Replacement Branch 
O, read by E. A. Rundquist. ` 


"m Readers are invited to send notes for this section 


D 


Ш 


to the Editor, EDUCATIONAL 
ч Psvcrorocicar, Measurement, 917 Fifteenth Street, N.W., Washington 5, 
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The papers presented at the meeting and the discussion concern- 
ing them are to be published by the University of Maryland. Copies 
of the proceedings may be ordered from Professor G. A. Kelley, De- 
partment of Psychology, The University of Maryland, College Park, 
Maryland. 


The preparation of a comprehensive handbook on educational 
measurement has been undertaken under the sponsorship of the 
Committee on Measurement and Guidance of The American Count 
on Education. The board of editors is composed of W. W. Cook, 
John C. Flanagan, E. F. Lindquist, chairman, Irving Lorge, т.к. 
McConnell, Philip J. Rulon, Donald J. Shank, John М. Stalnaker, 
Ralph Tyler, Kenneth Vaughn, and Ben D. Wood, with the chairman 
serving as editor-in-chief. Each of the twenty chapters is to be Piy 
pared by a specialist in the particular field covered, assisted Ў 
several collaborators. A production schedule has been outlined c? 
ing for sending final copy to the printers July 1, 1947. 


An Evaluation Service Center has been established at Syracus® 
University, with Professor Maurice E. Troyer as director. е р 
poses of the Center are (1) to help faculty members in their effort 5 
improve appraisal of student progress, (2) to assist faculty membe? 
їп constructing tests, (3) to make analyses of tests, (4) to keeP re 
make available to staff members an up-to-date file of sample publish?” 
and unpublished tests in the various subject areas, (5) to keeP i 
to-date and make available to staff members a library of reference 
on problems and procedures of measurement and evaluation in hight 


T or 
education, (6) to conduct seminars in problems of evaluation ү 
staff members, departmental assistants and scholars intereste ter- 
systematic and comprehensive study of test construction and JP 


pretation, (7) to assist with research (8) to encourage study an 
publication, by faculty members, of new and better methods he 


n = n е 
appraisal and instruction, and (9) to serve, through the staff of 


EE asis» 
Center and other members of the University Faculty on a tee b ob- 


as consultant to other colleges and universities in the area оп P 
lems of appraisal and instruction. 


. en 
The firm of Richardson, Bellows, Henry and Co., Inc., has bed 
established at 56 Beacon Street, New York, to conduct surveys ko 
research on problems of selection, placement training, and emP loy 
morale for business and industrial concerns, Tt will do job analy 
from the standpoint of qualification requirements and training E ter 
design application blanks, recommendation blanks, controlle n nd 
viewing procedures, aptitude tests, information tests, interest brit 
personality tests, and training outlines and manuals; develoP he 
rating systems and systems for combining scores (by means © ap 
usual multiple-correlation validation studies); make clinic? esi£? 
praisals of executive personnel; conduct attitude surveys cen 
personnel record systems and personnel statistical reporting sys 


i 
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and make over-all surveys of personnel programs. The member- 
ship consists of Roger M. Bellows, Francis F. Bradshaw (President), 
Edward E. Cureton (Secretary-Treasurer), Douglas Н. Fryer, Edwin 
R. Henry, Hermann H. Remmers, Marion W. Richardson (Chairman 
of the Board of Directors), Carroll L. Shartle, and Robert J. Wherry 
(Vice President). 


A counseling center has been established at the University of 
Chicago under the jurisdiction of the Dean of Students, Carl R. 
Rogers, to provide the services enumerated below. The volume of 
these services will be governed in accordance with the best interests of 
a sound program of professional education and research, carried out 
in cooperation with various interested departments and schools. 


Services 


1. To provide adjustment counseling to students, veterans, in- 
dustrial workers, and other individuals and groups. 

To provide a diagnostic service, using tests, interviews, and 
other techniques. 

To refer individuals to appropriate University services and 
agencies. 

To assist in the coordination of specialized counseling functions 
on the campus. 

To promote the development of in-service training programs 
oo those groups interested in improving their counseling 
skills. 


ona W. Blaesser, Treasurer of the American College Per- 

a nel Association and formerly of the University of Wisconsin, is 
w Assistant Dean of Students and Director of the Counseling 
enter at the University of Chicago. 


Pr Lieutenant Colonel J. P. Guilford has returned to the position of 
: ы essor of Psychology at the University of Southern California 
A. almost four years with the Army Air Forces. In his last assign- 
А nt he was Chief of the Department of Records and Analysis of the 
fien School of Aviation Medicine, Randolph Field. The Depart- 
К mes of Records and Analysis fell heir to the accumulated answer 
Unit 5, card records and data from all the examining and research 
San aircrew classification. Besides turning out a number of final 
on P. S the organization completed the writing of a 29-chapter volume 

rinted Aircrew Classification Tests, one of 15 volumes which are 


Scheduled to be writ | 
ме t th Its of th 
edule go be wri en to report the results of the AAF Psychological 


RR коч С. Taylor has been appointed director of the W. Е. 
ne ae SS for Community Research of Kalamazoo, Michigan. 
“ш bil e major objectives of the Institute is to investigate the 
ability of opportunity for gainful employment: its relationship 
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to the aptitudes, skills and interests of people; and its relationship to 
the satisfactions, monetary and otherwise, which people desire to 
obtain from their jobs.” Dr. Charles C. Gibbons is leaving his posi- 
tion as Director of Personnel Research of the Owens-Illinois Glass 
Company to become Assistant Director of the Institute. 


Professor John M. Stalnaker has left his position with the College 
Entrance Examination Board and Princeton University to become 
Dean of Students and Professor of Psychology at Stanford Univer- 
sity. Mr. Henry Chauncey has been appointed Associate Secretary 
and Professor Harold O. Gulliksen has been appointed Research 
Secretary of the College Entrance Examination Board. 


. -— 
Lieutenant Commander D. D. Feder has returned to his dd 

position as Executive Officer and Supervisor of the Illinois State M 

Service Commission. His last billet in the Navy was that of Offic 


in-Charge, Radio Material Unit, Test and Research Division, Burea 
of Personnel. 


[2 


1 


Lieutenant Colonel Paul Horst 


Procter and Gamble Company, wh 
Research. 


has left the Army to return to h 
ere he is now Director of Person 
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Berdie, Lt. R. F. “Range of Interests.” Journal of Applied Psychology, XXIX 
(1945), 268-281. свега 
An interest scale based on 22 items was found to differentiate clearly, “ou 
recruits who could be expected to adjust to military training and those w! ao 
not. As against the orally presented list, the printed list proved of greater € Say 
ence and objectivity. Age and educational factors must be taken into account H the 
analysis of the results. Supplementing the psychiatric screening technique anc, 


: À т á y r djust- 
interview, this range of interests test offers a satisfactory method of predicting 20) 
ment. Vernon S. Tracht. 


———— 


a Test 
Challman, Robert С. “The Validity of the Harrower-Erickson Multiple Choice » 
as a Screening Device.” Journal of Psychology, XX (1945), 41-48. . device 
The Harrower-Erickson Multiple Choice Test was designed as a selective ас 
for military use. The Procedure consists of offering subjects 10 choices for of indi- 
the 10 Rorschach cards. Half of the choices are considered representative O f che 
viduals suffering from mental abnormalities. If the subject considers none ased 
choices applicable, he is advised to submit an alternate. The critical scare E sired. 
on 4, 5, or 6 abnormal responses, depending upon the degree of selectivity ba as 
Three methods of Scoring were suggested. In Method I all alternates are c am by 
abnormal; in Method II alternates are scored abnormal only when characterize оз 
Poor form or when bizarre in content; in Method III abnormal and alternate resp 
are weighted, Harrower-Erickson found the Procedure valuable as a screening qi, 
vice. However, later studies, including the one described in this article, do not the 
cate a sufficiently sharp distinction between the responses of the normal an be 
abnormal to warrant the a 


` to 
1 the acceptance of the method as more than an auxiliary 
used with a personality inventory. Helen Heath, 


Forlano, G. and Kirkpatrick, Е. Н. "Intelligence and Adjustment Measuremsex ry 


the Selection of Radio Tube Mounters.” 9 hology; 
(1945). 257 341 Ounters.” Journal of Applied Psyc 


This study was concerned wi 


nd 


1 $0! 
tal Ability, Form B; (2) the 80 еде 
scale of the Bell Adjustment Inventory; (3) We alienate, scala of the Washbi by 


Social Adjustment Inventory. The criterion used for the experiment was ratin: 4 

the supervisor in charge of the group. It is concluded that low intelligence zi atc 
tend to indicate poorer workers but average scores or above do not discri ge 
between “good” and “fair” workers, while scores in social adjustment do differ 8 
“good” and "fair" workers. A composite of in seg 


am TS. telligence and personality sith: 
therefore effective in predicting success of new tube mounters. Frances 5" 


—— 


” 

Geil, George A. “A Clinically Useful Abbreviated Wechsler-Bellevue Scale. 
nal of Psychology, XX (1945), 101-108, | 1а meet 
Selection of a shortened form of the Bellevue full scale, which woelige? 

requirements of the clinical situation for time economy, accuracy of in 


* Edited by Forrest A. Kingsbury. 
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rating, and diagnostic screening capacity, was made on the basis of analysis of test 
records of a group of 250 unselected cases examined by the Wechsler-Bellevue full 
scale at the Medical Center for Federal Prisoners, Springfield, Mo. Mean weighted 
scores for each of the 10 subtests of the scale, and mean total full scale weighted sub- 
test score were determined, and mean total weighted subtest scores for trial combi- 
nations of selected subtests were then computed. Trial combinations were retained 
which showed close alignment of mean total weighted subtest scores with that indi- 
cated for the full scale, and were tested for accuracy by comparison of calculated IQ 
scores on the combinations for each of the 250 cases with the actual full scale IQ 
scores. An abbreviated scale composed of the tests of Comprehension, Similarities, 
Digits, and Block Design, shows a correlation of .966 + .003 with the IQ's of the full 
scale, and is found to be reliable as a screening instrument. Frances Smith. 


Gulliksen, Harold. “The Relation of Item Difficulty and Inter-Item Correlation to 

Test Variance and Reliability.” Psychometrika, X (1945), 79-91. 

Under assumptions that will hold for the usual test situation, it is proved that 
test reliability and variance increase (a) as the average inter-item correlation in- 
creases, and (b) as the variance of the item difficulty distribution decreases. As the 
average item variance increases, the test variance will increase, but the test reliability 
will not be affected. (It is noted that as the average item variance increases, the 
average item difficulty approaches .50). In this development, no account is taken 
of the effect of chance success, or the possible effect on student attitude of different 
item difficulty distributions. In order to maximize the reliability and variance of 
a test, the items should have high intercorrelations, all items should be of the same 
difficulty level, and the level should be as near to 50% as possible. (Courtesy 
Psychometrika.) 


Hildreth, Lt. H. M. “Single-Item Tests for Psychometric Screening.” Journal of 

Applied Psychology, XXIX (1945), 262-267. 

This describes how a series of 10 Single-Item Tests, selected from 30 experimen- 
tal items from well-known mental tests by a special scoring technique, were devised 
for use in the Navy's psychometric screening program. A successful response to 
any one of these tests indicated mental ability above the minimum—M.A. of 11 
years—arbitrarily selected by naval officials. Taking from 1 to 20 minutes per man 
to administer, and not requiring optimal testing conditions, they were standardized 
on 1500 cases; and, in a 3 months' trial period, exhibited an error in the selection of 
recruits of only 1/10 of 1 per cent. Vernon S. Tracht. 


Hult, Esther. "Study of Achievement in Educational Psychology." The Journal of 
Experimental Education, XIII (1945), 174—190. 

he purpose of this study was to find the relationship between practice teaching 
success and measures of ability considered prerequisite for this success as determined 
by varioys tests. The criteria were (1) practice teaching marks, and (2) ratings by 
supervisors. No significant relationships were found between practice teaching marks, 
general knowledge and mental ability. The multiple correlation between the several 
factors and the criterion was not high enough for individual prediction but there was 
a significant relationship between success in teaching and the grade point average. 
Shortly before the end of the semester the practice teachers rated their theory course 
and teacher, and practice course and teacher. Of the two courses, they tended to 


Tate their practice course higher; and of the two teachers, their practice teacher 
lower. Howard M. Schwman. 


Johnson, Palmer O. and Tsao, Fei. “Factorial Design and Covariance in the Study 

of Individual Educational Development." Psychometrika, X (1945), 133-162. 
: his is the report of the application of the principles of factorial design to an 
investigation of individual educational development. The specific type of factorial 
design formulated was a 2Х3Х3 х3 arrangement, that is, the effect of sex, grade 
location, scholastic standing, and individual order, singly and in all possible combi- 
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, a 
ions was studied in relation to educational development as measured by рее 
Tuis of Educational Development. An application of the covariance mel desea 
introduced which resulted in increased precision of this type p eid 
by significantly reducing experimental error. The two concomi ry шн en 
to increase the sensitiveness of the experiment were initial status of ea anil туо 
opment and mental age. Without these statistical controls all E SEMEN 
first-order interactions would have been accepted as significant. it ted significant 
sex (doubtful), scholastic standing, and individual order demonstra = вет 
effects. The chief beauty of the analysis of variance and covariance as эре 
part of a self-contained experiment is demonstrated in the complete nee 2 со been 
of the data. The statistical utilization of the experimental results | А АПУШ 
developed for purposes of estimation and prediction. The seca ag s ut of ins 
is being continuously required to develop and analyze experimenta [ ШЫ nance 
creasing complexity since the introduction of the analysis of variance an АНЫП i 
The mathematical formulation and solution of the problem of this inves Station 
carried out. The methods illustrated and explained in this study, and mo шүнүн 
and extensions of them are capable of very wide application. The general p aka 
can be used to various degrees and in a number of ways. (Courtesy Psychom 


__ 


131. 
Kaitz, Hyman В. “A Note оп Reliability.” Psychometrika, X (1945), 127 


А 1 EI M s ework 
__A formula for internal consistency reliability is developed within the fram 
of the analysis of variance. The test items are 


assumed to be homogeneous, p m 
have any weights. Data needed for computation are the student test scores E this 
total number of items answered so as to have the same weight. It is shown th ome 
formula reduces to the Kuder-Richardson for item weights of one and zero. 
empirical validation is offered. (Courtesy Psychometrika.) 


factors 

Martin, Howard G. “The Construction of the Guilford-Martin Inventory onie 

:G-A-M-I-N." Journal of Applied Psychology, XXIX. (1945), 298-300. essure 

Factor analyses have isolated five new temperament traits, (G) general T interz 
for overt activity, (A) social ascendancy, (M) masculinity of attitudes an were 
ests, (I) self-confidence, (N) lack of nervous tension. Over 300 items, ans stu- 
“Yes,” “No,” or “?” were administered to 250 men and 250 women, all collegen t 
ges of 19 and 30. Items, shown by factor and item analy: Four 
have heavy loadings in à trait, were used on the preliminary scoring key. inary 
hundred tests taken b men were scored with this prelim 
key, and the highest 1 
tors G, A, I, and N. 
lowest 100 females. 
Howard M. Schuman. 


pili- 
Newman, Joseph. “The Prediction of Shopwork Performance in an Adult Reha i 
tation Program. The Kent-Shakow Industrial Formboard Series." Psy 

cal Record, V (1945), 343-352. per 
ears) 


S ccor f^ 
2 ts were also ranked in ascending order а dete 
to total time score on the K-S Formb in Y 


1 to ^ 
h 2s Of: oard. Formb sults were studie Fori 
Беат relationship to shopwork та 
аг Den 


k 25 1! in 
T the Formboard is a total time а shop тай 
trachoric) of .76 was found betwee ances sm 
thinking, and total time scores. 
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Peel, E. А. “On Identifying Aesthetic Types.” British Journal of Psychology, 
XXXV (1945), 61-69. 

This paper outlines a method for estimating aesthetic preference with reference 
to artistic quality rather than to temperamental traits involved. Items were arranged 
by expert judges according to aesthetic criteria and the subjects’ orders of aesthetic 
Preference were compared with these criteria by means of correlation. „The correla- 
tions were then analyzed into factors characterizing the group of subjects and the 
criteria. Three different matrices of correlation coefficients were obtained: correla- 
tions between orders of “liking” for the subjects, correlations between the orders of 
“liking” and the criteria, and correlations between the criteria. Frances Smith. 


Rabin, Albert I. “The Use of the Wechsler-Bellevue Scales with Normal and Abnor- 
mal Persons.” Psychological Bulletin, XLII (1945), 410-422. 
Findings of various investigators employing the Wechsler-Bellevue Scale are 
coordinated and summarized and suggestions are offered for future treatment of the 
data rapidly being accumulated. Correlations between this scale and other tests of 


analysis because of the functional unity of its subtests; and the possibility of achiev- 
ing a diagnostic tool in individual cases through more effective control of major 
actors involved is discussed. Retest data with clinical material and miscellaneous 


Sarason, E. and Sarason, S. “A Problem in Diagnosing Feeble- indedness.” Jour- 
nal of Abnormal and Social Psychology, XL (1945), 323-329. 


tioning as part of a continuous behavioral sequence; (5) integration of information 
obtained from the various tests. Need for care in acceptance of numerical scores in 
doubtful and near-borderline cases is illustrated by presentation of the complete 
psychological report of a particular case. Frances Smith. 


Shakow, D., Rodnick, E. H., and Lebeaux, T. “A Psychological Stùdy of a Schizo- 
phrenic: Exemplification of a Method.” Journal of Abnormal and Social Psy- 
chology, XL (1945), 154-174, 

A collection of 8 psychological devices: (1) Stanford-Binet or Wechsler-Bellevue 
Intelligence Scale; (2) Rorschach Test; (3) Association Test; (4) Pinboard Aspira- 
tion Test; (5) Thematic Apperception Test; (6) Targetball-Thematic Test; (7) 
Pursuitmeter-Stress Test; and (8) Picture-Frustration Test were employed as one 
aspect of a comprehensive study of neuropsychiatric patients who had been in service 
in the armed forces, The specific aim of the psychological analysis was to construct 

: individual profiles and also to differentiate patient groups from each other and from 

Dormal groups. One case was reviewed in detail. Helen Heath, 


Thurstone, DSL CA. Multiple Group Method of Factoring the Correlation Matrix." 
Psychometrika, X (1945), 73-78. 
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ї i Н n : = re 
more interesting case in which a number of constellations are selected from the x 
lation matrix at the start. . The result of this method of factoring is a ае 

F which satisfies the fundamental relation FF^ = Р. (Courtesy Psychometriva- 


Walker, К. F., Staines, R. G., and Kenna, J. С. “The Influence of Scoring Мейо, 
upon Score in Motor Perseveration Tests.” British Journal of Psycho 
General Section, XXXV (1945), 51—60. уега- 
Spearman based his theory of mental inertia on the results of motor pers idate 

tion tests. Certain features in the construction and scoring of these tests inva Дер, 

these results. The two types of these tests are (1) creative effort, and Qua (c) 

nation. The five methods of scoring these tests are: (a) X - Y, (b) x/ tasks, 

X-X/X, (d) X+Y/2XY, and (e) E/A where X and Y are two interfering огге- 

E is the expected score, and A the actual score. Methods (а), (b), and (c) corre- 

late highly with each other but low or negatively with methods (d) and (e) when 

man’s general interference factor, found when method (d) is used, disappears vities 
method (e) is used. When initial differences in ease of performing the two adit in 

are great, the creative effort test using method (d) does not measure diffi i 

alternation but only the difference in ease of performance. The initial 


ri 
A а 0 
ease of performance is not related to ease of alternation of the two activities. H 
M. Schuman. 


j А е ior of 
Werner, Heinz (with the collaboration of Doris Carrison). “Perceptual Behavior 


ans 
Brain-Injured, Mentally Defecti i : D i 1 Study by y 
Ena hak y Defective Children: An Experimenta XXXI (1949) 


шет nique.” Genetic Psychology Monographs, deed 
Experimental analysis of perceptual and conceptual behavior of brain сход 

and non-brain-injured subnormal children of comparable mental ages WaS C° e were 
- by means of the Rorschach technique. Significant differences in терот, ten 
ш behavior of brain-injured children being characterized by disinteg" сто 
сатан, forced responsiveness to sensory stimulation, lack of affective m of thes 
Séponits is dite ү сано, eed and perseverations. шү КИШ 
larly formed groups of child of previous studies including Ern піп] 9160 тыр f 
ШИ cereal Ер i dren, work on the Rorschach test with brain Ae Just! fe 

ehavior traits of bns responses to the Rorschach test. Character. Fr 

Smith. rain-injured children are deduced from this analy 


